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ABSTRACT 


Understanding  treatment  dropout  in  outpatient  mental  health  services:  An 
economic  framework  exploring  the  relationship  between  patient 
satisfaction  and  appointment  noncompliance 

Rhonda  Gillespie  Ozanian 

Noncompliance  in  the  form  of  treatment  dropouts  is  a  major  problem  across  outpatient 
mental  health  settings  and  can  range  from  40-50%  of  all  clients.  Previous  research  has 
overlooked  the  impact  on  resource  use  and  research.  Economic  theory  suggests 
noncompliance  is  consumer  signaling  about  patient  preferences.  The  mental  health  industry 
standard  for  evaluating  patient  preferences  is  the  patient  satisfaction  measure.  This  study 
examined  the  relationship  between  treatment  noncompliance  and  patient  satisfaction 
hypothesizing  that,  after  controlling  for  sociodemographic  and  health  status  factors,  patient 
satisfaction  will  predict  duration  of  mental  health  treatment.  Using  1996-1999  data  from  the 
Agency  for  Healthcare  Research  and  Quality  Medical  Expenditure  Panel,  a  satisfaction 
measure  was  constructed  and  regression  models  developed  to  examine  the  impact  of 
satisfaction  on  treatment  duration  for  patients  having  at  least  one  mental  health  visit. 
Survival  analysis  was  used  to  examine  the  hazard  rate  of  patients  having  high  versus  low 
satisfaction.  Major  findings  suggests  individuals  satisfied  with  and  having  confidence  in  the 
provider  were  significantly  more  likely  to  have  a  subsequent  visit.  The  importance  of 
satisfaction  in  predicting  treatment  continuation  was  not  constant  with  respect  to  number  of 
visits  or  patient  sociodemographic  characteristics.  Satisfaction  was  strongly  predictive  of 
continuation  for  college  educated  and  individuals  with  a  mental  health  diagnosis,  but  less 
significantly  important  for  African  Americans,  high  school  educated  and  individuals  age  25- 
44.  Practical  aspects  of  care  such  as  access  were  significantly  more  important  for  Hispanics 
and  publicly  insured.  Evidence  suggests  there  is  a  distinct  group  of  consumers  who  seek  and 


are  satisfied  with  consultation  only.  MEPS  design  measures  satisfaction  with  ‘usual  source  of 
care’  lacks  specificity  and  does  not  reflect  satisfaction  specifically  with  mental  health  care. 
The  role  of  treatment  preferences  needs  to  investigated  further.  Patient  satisfaction  measures 
require  redesign  to  account  for  opportunity  costs  and  preferences.  Preference  measurement 
techniques  may  be  are  better  suited.  Treatment  noncompliance  should  be  the  gold  standard 
on  which  the  validity  of  satisfaction  and  preference  measures  are  based. 
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CHAPTER  1  Introduction 

Current  prevalence  rates  indicate  that  about  twenty  percent  of  the  U.S.  population 
are  affected  by  mental  disorders  in  a  given  year,  yet  two-thirds  never  seek  treatment  (U.S. 
Department  of  Health  and  Human  Services,  1999).  In  the  last  two  decades  researchers 
and  policy  makers  have  been  interested  in  identifying  and  removing  barriers  to  allow 
greater  access  to  mental  health  care.  However,  even  with  the  availability  of  health 
insurance  and  publicly  subsidized  care,  many  Americans  are  reluctant  to  make  use  of  the 
outpatient  mental  health  care  system  (Zuveckas,  2001).  Certain  groups  such  as  African 
Americans,  the  elderly,  the  uninsured,  and  individuals  with  fewer  years  of  education 
seem  particularly  prone  to  underutilize  outpatient  care,  either  by  never  entering  care,  or 
by  leaving  treatment  early  (Swartz  et  al.,  1998). 

The  focus  of  this  dissertation  research  is  on  persons  with  a  mental  health  disorder 
who  seek-but  do  not  complete  outpatient  mental  health  treatment.  Of  those  who  seek 
care,  one  can  expect  40-50  percent  will  terminate  from  care  early-many  after  only  the 
first  visit  (Wierzbicki  &  Pekarik,  1993;  Killapsy,  Banerjee,  King,  &  Lloyd,  2000). 
Treatment  dropouts  in  mental  health  are  persistently  high,  distort  important  research 
findings,  limit  the  effectiveness  of  psychological  treatments,  and  are  a  major  source  of 
economic  inefficiency  (PhiUipson  &  Hedges,  1998;  Ellickson,  Stern,  &  Trajtenberg,  1999, 
Hughes,  Bagust,  Haycox,  &  Walley,  2001).  Patients  who  leave  treatment  early  do  so 
presumably  against  medical  advice  and  are  systematically  deemed  treatment 
noncompliant  (Donovan,  1995).  Treatment  noncompliance  is  defined  as  the  failure  of 
the  patient  to  follow  through  with  treatment  regimens  or  the  degree  to  which  patient 
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behavior  is  congruent  with  treatment  recommendations.  There  are  many  ways  in  which  a 
patient  might  be  perceived  as  noncompliant  with  treatment.  In  the  context  of  this 
research  use  of  the  term  will  be  limited  to  appointment  noncompliance,  or  not  following 
through  with  appointments  for  mental  health  visits. 

Economic  theory  suggests  that  noncompliance  is  a  choice  consumers  make  about 
consumption  of  health  care  based  on  valuations  of  the  costs  and  benefits  of  care,  among 
other  factors  (Lee,  1995;  Hughes  et  al.,  2001).  A  universal  measure  that  seeks  to  reveal 
patient  valuations  and  experiences  with  their  health  care  is  the  patient  satisfaction  survey. 
If  treatment  noncompliance  is  economic  behavior  resulting  divergent  patient  valuations 
and  preferences  for  care  (Rao,  Weinberger,  &  Kroenke,  2000),  and  if  patient  satisfaction 
is  an  adequate  measure  for  revealing  these  valuations,  then  a  relationship  may  exist 
between  satisfaction  with  care  and  duration  of  treatment.  Using  1996-1999  data  from  the 
Agency  for  Health  Care  Research  and  Quality  (AHRQ)  Medical  Expenditure  Panel 
Survey  (MEPS),  I  will  to  examine  the  relationship  between  treatment  compliance  and 
satisfaction  with  outpatient  mental  health  care  predicting  that,  after  controlling  for 
sociodemographic  and  health  factors,  satisfaction  with  care  will  be  associated  with 
duration  of  treatment. 

What  are  the  implications  of  treatment  noncompliance  in  mental  health  care?  First, 
by  illuminating  only  our  success  with  those  who  comply,  noncompliance  has  globally 
distorted  the  industry’s  understanding  about  the  effectiveness  of  treatment.  Treatments 
which  demonstrate  efficacy  in  clinical  studies  do  so  among  a  select  group  of  patients, 
who  themselves  have  remained  in  the  trial  (I.  M.  Anderson  &  Tomenson,  1995).  By 
selecting  into  research  findings  only  those  who  comply,  clinical  research  can 
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overestimate  the  effectiveness  of  treatments  (Philhpson  &  Hedges,  1998).  When 
treatments  deemed  efficacious  in  clinical  studies  are  deployed  to  practice  settings  their 
effectiveness  becomes  further  diluted  by  the  effects  of  noncompliance  (G.  Simon, 
Wagner,  &  Vonkorff,  1995).  Perhaps  this  is  one  reason  why  the  effectiveness  of  mental 
health  treatments  has  been  slow  to  catch  up  with  the  growing  popularity  of  psychotropic 
medications  where  in  research  compliance  rates  are  a  critical  study  outcome.  Although 
rates  of  mental  health  treatment  have  increased  by  48  percent  since  1988,  the  number  of 
and  spending  for  psychotherapy  visits  is  declining  (Olfson  et  al.,  2002).  In  its  place 
spending  on  psychotropic  medications  has  doubled  (Glied,  Frank,  McCormack,  & 
O'Malley,  2001;  Zuvekas,  2001). 

A  second  implication  is  that  treatment  noncompliance  generates  inefficiencies  both 
in,  and  beyond,  the  health  care  system.  From  the  health  care  perspective,  reduced 
treatment  effectiveness  can  result  in  subsequent  additional  diagnostic  and  treatment 
procedures.  The  severity  of  disease,  cost  of  treatment,  and  opportunity  of  the  doctor 
determine  whether  savings  from  delivering  less  treatment  offsets  the  cost  from  increased 
morbidity  (Cleemput,  Kesteloot,  &  Degeest,  2002).  There  are  significant  indirect  costs 
from  treatment  noncompliance  that  occur  outside  the  health  sector.  An  untreated  or 
partially  treated  mental  health  problem  can  create  family,  social  and  legal  problems  that 
spillover  into  the  community  and  affect  costs  in  the  public  sector.  Although  some 
research  has  been  done  to  estimate  these,  the  costs  are  not  identified  in  the  majority  of 
mental  health  economic  evaluations-  often  because  prices  per  se  do  not  exist  for  these 
kinds  of  public  sector  costs  (Drummond,  O'Brien,  Stoddart,  &  Torrance,  2000). 
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A  third  implication  of  noncompliance  is  that  it  reduces  patient  welfare  by  preventing 
patients’  from  receiving  maximum  treatment  benefits  (Killapsy  et  al.,  2000).  This  is 
because  noncompliance  with  treatment  results  in  an  insufficient  level  of  medical  care 
being  delivered  than  is  needed  to  obtain  the  desired  health  effects.  In  spite  of  third  party 
payer  criticisms  about  the  effectiveness  of  many  psychotherapy  treatments,  it  is 
nonetheless  accepted  that  psychotherapy  methods  alone  or  in  combination  with 
medication,  case  management  and  other  forms  of  treatment,  are  effective  at  reducing  the 
symptoms  of  mental  health  disorders,  improving  the  quality  of  life  for  patients,  and 
providing  a  cost-effective  menu  of  treatments. 

Patient  acceptance  of  the  available  psychological  treatments  and  the  lack  of  industry 
knowledge  regarding  the  value  patients  place  on  treatments  are  major  concerns.  Equally 
disturbing  is  that  the  mental  health  industry  may  not  recognize  the  magnitude  of  the 
problem  (Howard,  Davidson,  O'Mahoney,  Orlinsky,  &  Brown,  1989;  Mueller  &  Pekarik, 
2000),  or  the  impact  it  has  on  advancing  the  field  of  psychological  treatments.  Those 
who  drop  out  from  treatment  are  rarely  asked  why  since  they  disappear  from  the  clinical 
setting  and  become  difficult  to  access  (Pekarik,  1992).  Whether  patients  do  not  improve 
because  they  drop  out  of  care,  or  whether  they  drop  out  of  care  because  they  do  not 
improve  is  an  important  distinction.  If  the  reason  for  dropping  out  is  dissatisfaction  with 
the  treatment,  then  word-of-mouth  effects  could  be  an  invisible  but  devastating 
contributor  to  the  stigma  of  mental  health  care  (Jones  &  Sasser,  1995;  Takeuchi,  Sue,  & 
Yeh,  1995). 

Measuring  patient  satisfaction  is  regarded  as  one  of  the  principal  methods  for 
obtaining  patient  evaluations  of  care  across  health  and  human  service  settings  (Carr-Hill, 
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1992;  Avis,  Bond,  &  Arthur,  1997;  Williams,  Coyle,  &  Healy,  1998).  Although  current 
measures  of  satisfaction  tend  to  be  conceptualized  globally  and  lack  an  agreed  upon 
theoretical  base,  the  possibilities  as  a  window  into  the  patients  attitudes,  beliefs  and 
preferences  for  care  remains  (Lebow,  1982;  Carr-Hill,  1992).  Compliance  on  the  other 
hand  is  a  clear  behavioral  measure  of  patient  valuations  and  abilities  to  follow  through 
with  care  (Phillipson  &  Hedges,  1998;  Ellickson  et  al.,  1999).  The  patient  satisfaction 
survey  has  served  as  a  proxy  for  patient  valuations  in  at  least  one  other  study  with 
researchers  finding  that  satisfied  patients  are  more  inclined  to  comply  with 
recommended  treatment  and  keep  appointments  than  dissatisfied  patients  (Carlsen  & 
Grytten,  2000).  Although  some  types  of  compliance  are  difficult  to  measure  such  as 
medication  compliance,  noncompliance  with  psychological  treatments  is  clear  and  can  be 
independently  verified  through  missed  appointments  and  early  termination  (Lebow, 
1982). 

For  this  research  mental  health  care  is  conceptualized  as  a  subspecialty  of  health  care. 
The  empirical  literature  to  support  the  study  comes  from  both  health  care  and  mental 
health  research  and  the  theoretical  model  originates  in  economics.  The  major  constructs: 
treatment  compliance  and  patient  satisfaction,  are  operationalized  similarly  across  both 
health  and  mental  health  settings.  Characteristics  of  treatment  and  the  range  of  patient 
responses  in  terms  of  economic  behavior  is  similar.  There  are  some  additional 
distinctions  with  regard  to  terminology.  The  terms  ‘treatment  noncompliance’, 
‘treatment  dropout  ’  and  ‘early  termination’  have  the  same  meaning,  but  the  terms  vary 
by  literature.  In  the  mental  health  practice  literature  the  terms  are  ‘treatment  dropout’  or 
‘non-adherence’.  In  the  economic  and  health  care  literature  the  term  is  ‘treatment 
noncompliance’  or  ‘noncompliance’.  Because  ‘level  of  use’  serves  as  a  proxy  for 
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treatment  noncompliance,  there  are  additional  terms  to  clarify  from  the  mental  health 
utilization  literature.  The  terms  ‘demand’,  ‘patterns  of  utilization’,  and  ‘level  of  use’  are 
used  to  refer  to  the  duration  of  mental  health  treatment.  Other  terminology  is  generic 
using  ‘patient’,  ‘doctor’,  ‘treatment’  and  ‘health  care’  to  indicate  a  client  or  patient, 
physician  or  mental  health  specialist,  medical  or  psychological  treatment,  and  health  or 
mental  health  care,  respectively. 

In  both  the  mental  health  and  health  care  literature,  researchers  have  examined  patient 
characteristics  related  to  utilization  and  compliance  (Olfson  &  Pincus,  1994b,  1994a; 
Alpine  &  Mechanic,  2000),  patient  characteristics  relating  to  clinical  outcomes,  patient 
characteristics  relating  to  satisfaction  with  care  (Hoff,  Rosenheck,  Meterko,  &  Wilson, 
1999)  (Marshall,  Hays,  &  Mazel,  1996),  clinical  outcomes  relating  to  compliance,  and 
rliniral  outcomes  related  to  satisfaction  (Ankuta  &  Abeles,  1993)  (Plante,  Couchman,  & 
Hoffman,  1998)  (Lambert,  Salzer,  &  Bickman,  1998;  Pekarik  &  Guidry,  1999).  However, 
only  three  published  mental  health  studies  (Pekarik,  1992;  Brannan,  Sonnichsen,  & 
Heflinger,  1996;  Mueller  &  Pekarik,  2000)  have  examined  the  influence  of  satisfaction  on 
treatment  compliance.  Pekarik  (1992)  found  differences  in  patient  satisfaction  between 
dropouts  and  completers  of  treatment,  but  employed  only  a  single  global  measure. 
Brannan  et  al.  (1996)  found  a  statistically  significant  relationship  between  the  patient- 
provider  relationship  and  length  of  treatment,  but  data  was  drawn  on  active  patients  at 
six  month  intervals  -  well  beyond  the  point  when  most  dropouts  leave  care.  Mueller  and 
Pekarik  (2000)  examined  the  relationship  between  expected  and  actual  duration  of 
treatment  finding  that  patients  attending  fewer  sessions  had  lower  satisfaction,  but  the 
focus  of  this  study  was  on  the  accuracy  of  therapist  versus  patient  expected  treatment 


duration. 
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By  adopting  an  economic  framework,  this  research  implicitly  challenges  the 
theoretical  models  which  have  previously  driven  research  on  health  care  utilization  and 
compliance,  such  as  the  paternalistic,  socio-behavioral  model  and  health  belief  models. 
The  paternalistic  model  has  a  long  tradition  is  medicine  and  is  based  on  the  deeply 
imbedded  philosophy  that  noncompliance  is  a  failure  on  the  part  of  the  patient  to  follow 
doctors  instructions.  The  doctor  is  the  ultimate  authority  for  deciding  what  is  best  for 
the  patient  and  noncompliance  results  from  patient  flaws  that  are  outside  the  bounds  of 
responsibility  of  the  doctor  (Trostle,  1988).  Paternalism  has  played  a  role  in  past  research 
less  as  a  theory  than  as  a  subtle  bias  limiting  research  only  to  that  which  reveals 
characteristics  of  the  patient  that  contribute  to  noncompliance. 

The  socio-behavioral  model  has  been  dominant  in  health  utilization  research. 
According  to  this  model,  utilization  of  health  care  results  from  an  interplay  between 
predisposing  and  enabling  factors  in  the  patient’s  environment  which  serve  to  either 
encourage  or  discourage  health  care  use  (R.  M.  Anderson,  1995).  Factors  which 
predispose  individuals  to  health  care  utilization  are  age,  race  and  gender,  while  factors 
that  enable  utilization  are  income  and  insurance.  A  second  influential  model  is  the  health 
beliefs  model  which  purports  that  individuals  decide  whether  or  not  to  use  medical 
services  given  their  beliefs  and  the  constraints  of  their  everyday  lives.  Health  beliefs 
affecting  decisions  to  seek  care  are  generated  from  beliefs  about  the  symptoms, 
perceived  benefits  of  treatments  and  experience  with  health  care  (Pescosolido  &  Boyer, 
1999). 
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A  major  criticism  of  these  three  models  is  that  they  overemphasize  patient 
characteristics  in  predicting  utilization  without  regard  for  how  patients  value  the 
treatment  in  the  first  place.  According  to  all  three  models  the  causes  of  treatment 
noncompliance  rests  primarily  with  the  patient  or  within  the  patient  environment. 
Increasing  compliant  behavior  means  altering  patient  beliefs  and  motivation,  and 
removing  environmental  barriers.  However  the  pervasiveness  of  treatment 
noncompliance  in  mental  health  that  occurs  independent  of  patient  population  or  clinical 
settings  suggests  the  problem  is  too  widespread  to  be  addressed  by  strategies  that  seek  to 
alter  patient  beliefs  about  the  benefits  of  treatment.  A  second  weakness  of  these  models 
is  that  they  do  not  allow  for  the  possibility  that  noncompliance  may  be  a  response  to 
factors  on  the  supply  side,  such  as  adequacy  of  the  treatments  themselves.  In  sharp 
contrast  according  to  the  rational  choice  theory  in  economics,  patients  do  not  fail  to 
comply-they  simply  choose  another  action.  If  patients  refuse  to  comply  with  treatment- 
it  is  the  adequacy  of  the  treatment-  not  the  motivation  of  the  patient  that  is  scrutinized 
(Donovan,  1995).  Rational  choice  models  have  been  previously  rejected  in  mental  health 
research  because  of  concerns  about  whether  individuals  with  a  mental  health  problem 
are  capable  of  rational  decisionmaking. 

There  are  several  areas  of  economic  theory  that  are  relevant  to  this  research.  First, 
utility  theory  also  known  as  demand  or  consumer  theory  describes  the  influence  of 
consumer  preferences  on  the  demand  for  mental  health  care.  Utility  is  the  satisfaction 
individuals  derive  from  the  goods  and  services  they  consume  and  is  based  on  preferences 
and  attitudes  toward  risk.  Consumers  make  economic  choices  that  maximize  utility 
within  budget  constraints  (Ellickson  et  al.,  1999).  Second  the  theory  of  revealed 
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preferences  describes  preferences  as  endogenous,  e.g.,  determined  by  individual 
characteristics  such  as  age,  race,  gender,  culture,  education,  and  shaped  thereafter  by 
experiences  (Ryan,  McIntosh,  &  Shackley,  1998;  Scott,  Watson,  &  Ross,  2002). 
According  to  both  utility  theory  and  theory  of  revealed  preferences  individuals  reveal 
their  preferences  by  economic  behavior  and  economic  behavior  is  determined  by 
preferences  and  constrained  by  income  and  price.  Choosing  whether  to  comply  with 
treatment  is  an  economic  choice.  Third  and  more  broadly  relevant  are  the  theoretical 
foundations  of  welfare  economics  which  explain  health  capital,  the  demand  for  medical 
care  as  inputs  into  household  production,  distributional  issues  that  result  from  market 
failure,  and  externalities  related  to  social  costs  and  benefits  of  mental  health  care.  The 
following  chapter  establishes  the  theoretical  framework  for  this  research  by  framing  the 
problem  of  noncompliance,  the  central  role  of  patient  preferences  and  the  theoretical 
underpinnings  of  patient  satisfaction  in  economic  terms. 
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CHAPTER  2  Theoretical  framework 
Economic  theory 

Economic  theory  has  yet  to  be  used  to  explain  noncompliance  in  outpatient  mental 
health  care.  Yet  it  is  potentially  more  forward  thinking  than  paternalistic,  socio- 
behavioral,  and  health  beliefs  models  for  several  reasons.  First  30  years  of  research  using 
the  latter  models  have  yielded  no  conclusive  findings  regarding  either  satisfaction  with 
care  or  treatment  noncompliance.  Second  economic  models  have  done  much  to  advance 
the  technology  of  pharmaceutical  medications  where  compliance  is  an  even  greater 
burden.  These  advances  have  been  achieved  by  understanding  noncompliance  is  an 
expression  of  consumer  signaling  about  preferences.  When  patients  are  not  happy  with 
the  costs  (side  effects,  pain,  administration,  inconvenience),  or  do  not  experience 
benefits  from  treatment,  they  choose  not  to  comply.  Finally  economic  models  consider 
both  demand  and  supply  side  factors  that  influence  patient  choices  about  compliance. 

For  this  study  patient  noncompliance  is  hypothesized  as  a  response  to  dissatisfaction 
with  some  aspect  of  outpatient  mental  health  treatment.  The  nature  of  the  dissatisfaction 
is  unknown.  However,  it  could  be  patient  valuations  of  the  costs  versus  benefits  of  care 
(Giuffrida  &  Gravele,  1998),  uncertainty  surrounding  the  physician’s  choice  of  treatment 
for  the  patient  (Charles,  Amiram,  &  Tim,  1997),  or  uncertainty  around  the  effectiveness 
of  medical  care  (Lee,  1995;  Ellickson  et  al.,  1999).  The  demand  for  health  care  is 
normally  approached  in  the  context  of  any  use,  and  less  so  in  the  context  of  level  of  use. 
However,  as  I  will  describe,  patients  may  be  unable  to  determine  how  a  medical 
treatment  will  improve  health  outcomes  until  they  have  obtained  information  through  at 
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least  one  visit  for  consultation.  Therefore  the  demand  for  any  use  reveals  litde  about 
demand  for  the  level  of  use  thereafter.  Health  care  is  unique  in  that  the  demand  for 
health  care  services  are  derived  from  the  demand  for  health  and  may  reflect  an 
individual's  value  for  health  more  than  preferences  for  any  particular  health  inputs 
(Grossman,  1972).  The  distinction  is  offered  to  illustrate  the  complexity  in  patient 
preferences  and  the  demand  for  health  care. 

This  section  will  first  address  consumer  demand  as  it  relates  to  health  capital, 
household  production,  cross  elasticity  of  demand,  risk  and  uncertainty,  and  compliance 
costs.  The  second  half  of  this  section  will  examine  supply  side  factors  that  influence 
demand  and  subsequent  compliance.  The  role  of  information  asymmetry,  adequacy  of 
the  technology,  benefits  of  treatment  and  value  created  will  be  discussed.  Finally,  this 
section  will  describe  how  treatment  noncompliance  and  patient  satisfaction  with  care  are 
studied  in  a  purely  economic  sense.  Understanding  the  demand  for  health  comes  from 
classic  economic  papers  of  Arrow  (1963)  and  Grossman  (1972).  From  their  research  on 
drug  therapies,  Hughes  et  al,  (2001),  Ellickson  et  al.,  (1999),  Giuffrida  and  Gravele, 
(1998),  and  Charles  et  al.,  (1997)  describe  the  relationship  between  the  effectiveness  of 
medical  care  and  its  effect  on  compliance  and  patient  welfare.  Lee  (1995),  Dardanoni  and 
Wagstaff  (1990)  and  Pauly  (1980)  discuss  the  role  of  information  and  uncertainty  in 
patient  decision-making  about  care  and  ability  to  evaluate  care.  Patient  valuations  of 
treatment  noncompliance  and  value  created  from  health  care  interventions  comes  from 
Drummond,  et.al.  (2000),  Hirth,  Bloom,  Chernow  and  Fendrick  (2000)  and  Singh, 
Hawthorne  and  Vos  (2001). 
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Demand  for  health 

Before  introducing  additional  economic  concepts  for  this  framework  it  is  important 
to  recognize  how  mental  health  care  is  viewed  within  the  larger  context  of  health  care. 
First,  there  are  substantial  social  benefits  from  the  provision  of  health  care.  Health  care 
generates  benefits  and  costs  that  are  greater  than  those  accrued  by  the  individual  patient 
(Blank,  2000;  Hirth  et  al.,  2000).  Referred  to  as  externalities,  this  phenomenon  exists 
when  costs  are  imposed,  or  benefits  conferred,  which  are  not  included  in  the  price  or 
cost.  It  is  part  of  the  market  failure  problem  that  characterizes  all  of  health  care,  but  is 
particularly  problematic  in  mental  health  because  of  the  widespread  social  benefits 
(Giuffrida  &  Gravele,  1998).  In  a  market-based  health  care  system,  where  only  price  and 
quantity  matter,  some  populations  will  be  unable  to  purchase  health  care  at  a  price  they 
can  afford.  The  free  market  is  thought  to  best  operate  without  interference,  but  when 
the  market  fails,  government  intervention  is  sometimes  justified.  To  correct  for 
externalities,  the  government  provides  subsidies  to  finance  health  care  benefits  that  the 
public  enjoys,  but  the  individual  would  not  be  willing  to  pay.  To  address  fairness  and 
equity  issues,  regulations  are  imposed  on  payers  and  providers  to  ensure  health  care 
access,  and  additional  subsidies  ensure  health  care  availability  to  populations  unable  to 
pay,  such  as  low  income  and  the  disabled. 

Consumers  themselves  require  health  care,  not  for  its  own  sake,  but  rather  as  inputs 
to  produce  the  commodity  good  health  (Grossman,  1972).  Health  is  an  investment 
consumers  make  to  produce  an  output  of  healthy  time,  which  is  the  amount  of  time  that 
can  be  spent  producing  money  earnings  and  other  household  commodities.  The  greater 
the  wage  rate,  the  greater  the  value  of  an  increase  in  healthy  time  (Luce  &  Elixhauser, 
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1990).  While  shifts  in  human  capital,  like  education,  change  the  efficiency  of  household 
and  labor  market  production,  changes  in  health  capital  affect  the  amount  of  productive 
time  lost  to  illness  (Grossman,  1972).  Mental  health  conditions  probably  create  changes 
in  both  health  and  human  capital.  It  is  the  fundamental  arrangement  of  these 
commodities,  not  the  medical  inputs  required  to  produce  them,  that  reflect  consumer 
preferences  and  make  up  arguments  of  the  utility  function  (Pollack  &  Wachter,  1975). 
All  individuals  are  assumed  to  be  utility  maximizers  in  that,  within  their  budget 
constraint,  they  will  maximize  the  expected  value  of  their  utility  (Arrow,  1963).  A 
potential  patients’  preferences  are  characterized  by  the  rate  of  substitution  between 
health  and  consumption  of  commodities;  the  more  an  individual  loves  health  the  more 
he  is  willing  to  sacrifice  consumption  of  other  commodities  for  health  (Lee,  1995). 
However,  not  all  individuals  hold  the  same  value  for  health  and  the  extent  to  which  they 
carry  out  actions  to  improve  health  will  depend  on  whether  they  value  health  in  the  first 
place  (Ellickson  et  al.,  1999).  Thus  patients  with  a  lower  value  on  health  would  not  be  as 
likely  to  place  health  care  ahead  of  inputs  to  produce  more  highly  valued  household 
commodities,  and  would  therefore  be  less  likely  to  comply  with  recommended  health 
actions  (Sherboume,  Hays,  Ordway,  DiMatteo,  &  Kravitz,  1992) 

The  demand  for  medical  care  to  produce  health  depends  on  cost,  utility  gained  from 
receiving  care,  and  the  degree  of  substitutability  with  other  products  (Ellickson  et  ah, 
1999).  The  demand  for  mental  health  care  is  known  to  be  price  sensitive,  such  that  small 
increases  in  price  make  big  differences  in  demand  (Horgan,  1983;  Taube,  Kessler,  & 
Burns,  1986;  K.  B.  Wells,  Keeler,  &  Manning,  1990).  This  has  been  thought  to  pose  a 
barrier  for  persons  without  insurance  especially  given  discrepant  use  among  low  income 
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groups  and  minority  populations.  However,  research  findings  do  not  necessarily  show 
that  even  in  the  presence  of  insurance  those  groups  will  use  anymore  care  (Swatrz  et  al., 
1998). 

The  demand  for  care  is  also  responsive  to  price  is  because  of  cross-elasticity,  or  the 
availability  of  less  costly  substitutes,  such  as  family,  friends  and  community.  Many 
consumers  simply  may  have  better  alternatives  to  mental  health  care,  but  this  does  not 
explain  why  individuals  would  seek  treatment  to  begin  with.  Economic  research  into 
non-compliance  with  drug  therapies  provides  a  helpful  framework  here.  Knowledge  on 
treatment  compliance  has  been  greatly  advanced  in  pharmacoeconomics  because 
researchers  target  the  inferiority  of  the  technology-not  the  patient.  If  patients  are  not 
happy  with  side  effects,  experience  discomfort  or  suffer  poor  outcomes  from  drug 
compliance,  then  the  drug-not  the  patient  needs  to  change.  Thus  great  strides  have  been 
made  to  improve  compliance  by  reducing  side  effects  and  improving  administration  to 
facilitate  greater  patient  compliance.  These  advances  reduce  the  non-price  costs  of  care 
borne  by  the  patient  which,  in  turn,  seem  to  improve  the  patient’s  compliance  with  the 
treatment. 

Patient  valuations 

Treatment  noncompliance  rates  reflect  patients’  valuations  of  particular  therapies 
(Ellickson  et  al,  1999).  For  patients,  compliance  is  not  an  issue,  they  do  not  obey  or 
disobey  the  doctor’s  orders  (Donovan,  1995).  Patient’s  choose  to  be  noncompliant  if  the 
costs  of  compliance,  such  as  increased  frequency  of  complications  or  morbidities, 
outweigh  the  benefits  in  terms  of  improved  quality  of  life.  Patients  make  a  personal 
tradeoff  between  efficacy  of  treatment  and  the  side  effects  it  generates  (Cleemput  et  al., 
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2002).  Factors  associated  with  compliance  rates  include  the  behavior  changes 
recommended,  complexity  of  the  regimen,  and  ease  to  which  the  patient  can  incorporate 
the  recommendations  into  their  daily  routine  (Miller,  Hill,  Kottke,  &  Ockene,  1997). 
Patients’  evaluations  of  the  effectiveness  of  treatment  and  severity  of  side  effects 
suggests  patients  respond  to  perceived  costs,  both  monetary  and  psychic  when  choosing 
whether  or  not  to  comply,  and  even  the  long  term  health  prospects  resulting  from 
complying  often  do  not  outweigh  the  more  immediate  concerns  with  side  effects, 
inconvenience  or  discomfort  (Ellickson  et  al.,  1999). 

This  is  related  to  the  notion  that  individuals  have  a  positive  time  preference, 
preferring  money  and  resources  now  as  opposed  to  later  in  order  to  benefit  from  them 
in  the  interim  (Birks  &  Buurman,  1994;  Drummond  et  al.,  2000).  Two  of  the  most 
important  patient  and  family  resources  needed  to  comply  with  treatment  is  time  and 
money  because  of  the  high  opportunity  costs  of  both  (Luce  &  Elixhauser,  1990; 
Ellickson  et  al.,  1999).  Opportunity  costs  are  the  value  of  foregone  benefits  when  a 
resource  is  not  available  for  its  best  use  (Drummond  et  al.,  2000).  The  impact  of 
monetary  costs  in  determining  mental  health  utilization  is  well  established,  and  can  be 
evaluated  directly  by  through  insurance  coverage,  premiums  and  out-of-pocket 
payments.  Time  costs  are  not  as  well  studied,  except  in  terms  of  productivity  (human 
capital)  losses  to  society.  However,  among  chronic  mental  health  conditions,  such  as 
depression,  an  individual  may  not  be  in  optimal  health,  but  not  impaired  enough  to  stay 
home  (Brouwer,  Koopmanschap,  &  Rutten,  1999).  Complying  with  complex  treatments 
must  then  compete  with  time  that  would  otherwise  be  devoted  to  wage  earning  (Singh  et 
al.,  2001). 
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Treating  noncompliant  patients  is  likely  to  be  more  costly  than  treating  compliant 
patients,  but  the  outcome  of  noncompliance  for  the  patient  is  less  certain  (Giuffrida  & 
Gravele,  1998).  From  the  patient’s  perspective,  noncompliance  may  temporarily  improve 
quality  of  life  as  the  individual  deliberately  adapts  treatment  to  fit  his  own  lifestyle,  or  it 
may  decrease  quality  of  life  through  reduced  effectiveness  and  increased  long  term 
morbidity  (Cleemput  et  al,  2002).  Giuffrida  and  Gravele  (1998)  explain  even  a  reduction 
in  treatment  effectiveness  depends  on  the  health  gains  that  would  have  been  derived 
from  treatment;  time  and  other  costs  resulting  from  compliance;  and  the  patient  s 
valuations  of  these  gains  and  costs.  Finally,  from  the  societal  perspective,  untreated  or 
partially  treated  mental  illness  resulting  from  noncompliance  with  treatment  creates 
resource  use  in  other  sectors  through  time,  financial,  and  emotional  costs  of  caretaking 
by  family  and  community,  and  such  things  as  emergency  housing,  social  services,  legal 
services  and  law  enforcement.  The  patient  does  not  take  these  negative  externalities  into 
account  in  his  decision  not  to  comply  (Cleemput  et  al.,  2002). 

Risk  and  Uncertainty 

An  individual  knows  best  how  improvements  in  health  status  affect  his  or  her  well 
being  and  his  willingness  to  pay,  but  there  is  uncertainty  regarding  the  diagnosis, 
treatment  that  may  be  needed  and  expected  outcomes.  Uncertainty  affects  the 
individual’s  expectations  for  health  outcomes,  beliefs  about  the  adequacy  of  medical 
treatment,  and  the  ability  of  the  doctor  to  achieve  those  outcomes.  Individuals  evaluate 
uncertainty  depending  on  preferences  for,  or  willingness  to  bear  risk.  Risk  aversion  is  the 
most  common  attitude  toward  risk-  most  individuals  prefer  a  certain  outcome  to  a  risky 
outcome  that  probabilistically  has  the  same  expected  value.  Dardanoni  and  Wagstaff 
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(1990)  describe  two  types  of  uncertainty.  Type  I  uncertainty  surrounds  the  individuals 
lack  of  diagnostic  information  to  accurately  assess  his  level  of  health  and  eventual  health 
outcomes.  Type  II  uncertainty  surrounds  the  effectiveness  of  medical  care.  Uncertainty 
in  medical  care  often  results  from  asymmetric  information  which  affects  both  patient 
and  provider  decision  making.  For  example  the  patient,  unlike  his  physician,  has 
imperfect  knowledge  about  the  effectiveness  of  different  types  of  medical  care. 

According  to  Lee  (1995)  a  patient's  willingness  to  comply  with  a  treatment  mainly 
stems  from  his  own  way  of  interpreting  medical  information  that  is  often  different  from 
the  doctor.  A  potential  patient  chooses  a  threshold  health  level  such  that  he  decides  to 
consult  a  doctor  only  when  his  health  falls  below  this  level  with  the  threshold  health 
level  determined  by  preferences,  income,  user  price,  and  medical  information.  The  more 
severe  the  illness,  the  lower  the  elasticity  of  demand  for  medical  care  (Lee,  1995). 
Crossing  the  doctor's  threshold  gives  the  patient  a  chance  to  obtain  objective 
information  about  the  pattern  and  magnitude  of  medical  care  needed  for  raising  the 
health  status.  This  information  consists  of  diagnosis,  and  identification  of  the  capacity, 
pattern  and  magnitude  of  the  medical  care  needed  to  improve  health  status.  Once  the 
patient  has  obtained  information  about  diagnosis  and  available  treatment,  there  is  still 
uncertainty  around  the  capacity  of  medical  care  to  improve  health  status,  and  a  decision 
must  be  made  about  whether  to  accept  the  treatment. 

Supply-side  factors 

While  the  patient  lacks  information  about  diagnosis,  treatment  and  outcomes,  the 
physician  does  not  have  perfect  knowledge  about  the  patient’s  personal  objectives, 
attitude  or  ability  to  pay  (Donovan,  1995;  Lee,  1995;  Ellickson  et  al.,  1999;  Frieman  & 
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Zuvekas,  2000).  Pauly  (1980)  points  out  the  doctor  may  be  well  informed  about  the 
technology  to  improve  health,  but  may  be  poorly  informed  about  the  patients  demand 
for  health  and  preferences  for  care.  Physicians  may  be  less  sensitive  than  patients  to  the 
effects  of  treatment,  opportunity  costs  and  preferences  for  risk,  and  may  underinvest  in 
gathering  the  types  of  information  about  patients  which  yield  the  best  fit.  Patients  are 
selected  into  treatments  by  their  doctors,  but  doctors  draw  upon  their  own  stock  of 
knowledge  and  understanding  when  deciding  upon  a  treatment  regimen,  and  do  not 
necessarily  prescribe  treatments  in  a  neutral  manner  (EUickson  et  al.,  1999). 

Freiman  and  Zuvekas  (2000)  explain  that  information  asymmetry  is  particularly 
problematic  in  mental  health.  The  accurate  diagnosis  and  treatment  of  mental  health 
problems  relies  heavily  on  the  interaction  between  patient  and  provider.  The  provider 
may  misperceive  patient  information  inferred  by  descriptions  of  emotions,  thoughts  and 
events,  and  interpretation  of  voice  and  visual  cues.  Patients  may  differ  in  their  ability  to 
effectively  communicate  symptoms,  and  may  misperceive  their  own  mental  state  and 
symptoms.  Providers  respond  to  patient  characteristics  such  as  age,  race/ethnicity, 
gender,  and  education  when  considering  patient  preferences  and  treatment  outcomes.  As 
a  result,  the  provider  may  apply  different  norms  in  assessing  symptoms  of  mental  health 
problems  according  to  the  gender  or  ethnicity  of  the  patient. 

Technology  influences 

Medical  technology,  both  its  efficacy  under  ideal  conditions  and  its  adequacy  in 
practice,  effects  decisions  about  compliance  (Hummel,  Rossum,  &  Verkerke,  2000). 
Health  care  is  described  as  a  set  of  technologies  that  probabilistically  span  zero  to  one.  A 
better  technology  is  one  with  a  higher  probability  of  choosing  the  correct  diagnosis  and 
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treatment,  and  that  offers  a  greater  chance  of  recovery  for  the  patient  (Leonard  &  Zivin, 
2000).  The  effectiveness  of  mental  health  care  is  often  complicated  by  known  difficulties 
with  the  reliability  of  diagnoses,  and  frequent  lack  of  consensus  about  etiology  and 
treatment  for  many  psychiatric  illnesses.  Treatments  are  sometimes  compulsory,  ill 
defined  and  many  have  not  been  proven  effective  (Evers,  Van  Wuk,  &  Ament,  1997). 

Berndt,  Bir,  Busch,  Frank  and  Normand  (2000)  estimate  through  expert  panel 
testimony  that  40  percent  of  medical  treatment  for  acute  stage  depression  involve 
treatments  with  litde  or  no  incremental  effectiveness  over  waiting  list.  In  an  earlier  paper, 
Berndt,  Busch  and  Frank  (1998)  found  all  the  major  treatment  technologies  (including 
medication)  for  the  average  care  of  less  severe  acute  stage  depression  offer  comparable 
expected  outcomes.  This  is  not  unique  to  mental  health  care.  During  the  1980’s  Rand 
Health  Insurance  Experiment,  individuals  with  better  insurance  used  more  health  care, 
but  had  health  outcomes  that  were  no  better  than  those  who  used  less  care.  Patients 
receiving  over  three  times  as  much  care  were  more  likely  to  survive  a  hospital  stay,  but 
by  nine  months  post  discharge,  both  groups  had  the  same  mortality  rate  (Cutler,  1996). 
The  implication  is  that  about  half  of  medical  progress  was  directed  at  keeping  patients 
alive  for  several  months,  but  not  for  as  long  as  a  year.  This  is  an  argument  about  the 
diffusion  of  technology  (Hummel  et  al.,  2000).  Technology  that  has  diffused  becomes 
widely  overused.  This  includes  use  for  those  who  would  benefit  a  lot  and  those  who 
would  only  benefit  a  little  (Cutler,  1996).  If  treatments  are  applied  to  those  who  benefit  a 
little  and  if  patients  do  assess  the  treatment  benefits  in  decisions  about  continuation, 
then  the  diffusion  of  treatment  technologies  could  be  a  contributing  factor  to  rates  of 
noncompliance. 
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Compared  to  other  medical  treatments,  the  effectiveness  of  talk  therapies  is 
sometimes  minimal,  in  that  many  conditions  are  not  responsive  to  the  available 
treatment,  and  unlike  the  rapidly  advancing  technology  in  other  health  sectors,  the  nature 
of  talk  therapy  has  simply  not  changed  much  over  the  years.  This  is  puzzling,  but  may  be 
related  to  the  level  of  public  protection  granted  to  the  mental  health  industry,  and  the 
resulting  lack  of  market  forces  to  reward  innovation  and  improve  technology.  As 
discussed  earlier  in  this  paper,  although  there  are  strong  societal  preferences  for  access  to 
mental  health  care,  these  do  not  seem  to  extend  to  maximizing  treatment  effectiveness. 
The  societal  costs  of  mental  illness  and  market  failure  in  the  provision  of  mental  health 
care  has  resulted  in  government  regulation  to  ensure  parity  in  insurance  coverage;  heavy 
subsidization  of  direct  costs  through  the  public  sector;  and,  routine  cross  subsidization 
of  operational  costs  from  resource-generating  departments  to  mental  health  units  within 
health  care  organizations  (Luce  &  Elixhauser,  1990).  These  actions  ensure  access,  but 
may  not  provide  adequate  incentives  to  improve  the  effectiveness  of  treatments. 

The  ubiquitous  nature  of  mental  health  problems  and  the  broad  impact  on  society 
makes  the  development  of  better  diagnostic  and  treatment  technology  difficult.  First, 
there  is  no  finite  criteria  for  defining  mental  illness  in  a  way  that  clearly  delineates  the 
boundaries  of  what  is  and  more  critically  what  is  not  a  mental  health  problem.  Therefore, 
the  scope  of  the  illnesses  that  are  diagnosable,  and  treated  with  the  current  technology 
continues  to  creep  outward  often  following  political  and  reimbursement  paths.  Second, 
although  some  mental  health  conditions,  like  depression,  are  thought  to  cause  greater 
functional  loss  than  many  physical  illnesses  (Singh  et  al.,  2001),  mental  health  conditions 
are  largely  chronic,  non-lethal,  and  even  successful  treatment  is  not  likely  to  produce  a 
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gain  in  terms  of  life  years  (Evers  et  al.,  1997).  Third,  important  costs  resulting  from 
mental  illness  are  accrued  outside  the  health  sector,  by  family,  community  and  public 
sector.  Conditions  such  as  schizophrenia  and  bipolar  disorders  are  thought  to  have 
especially  large  impacts  on  both  family  and  community  (Tarricone  et  al.,  2000). 

Benefits  of  treatment 

Nonetheless  mental  health  care  is  presumed  to  have  substantial  benefits.  According 
to  Drummond  (2000)  there  are  three  types  benefits  that  can  be  derived  from  a  health 
intervention:  improvements  in  health  status,  saving  of  resources,  and  creation  of  other 
value.  The  first  two  are  recognized  benefits  of  mental  health  care,  but  are  difficult  to 
value.  The  third  is  both  difficult  to  value  and  its  benefits  often  overlooked.  The  most 
important  outcome  of  changes  in  health  status  is  the  creation  of  healthy  time.  This  is 
measured  through  changes  in  productivity  and  with  the  exception  of  severe  mental 
illness,  these  improvements  are  gradient  as  discussed  earlier.  The  second  type  of  benefit 
is  the  saving  of  resources.  This  occurs  when  one  intervention  is  equally  effective  as  its 
alternative,  but  less  costly.  The  third  benefit,  creation  of  other  value,  is  thought  to  be  a 
centrally  important  outcome  of  mental  health  care,  but  little  is  known  about  how  much 
value  patients,  physicians,  payers,  and  the  public  place  on  these  intangible  benefits. 

Value  may  be  created  by  a  health  care  intervention,  but  not  necessarily  linked  to  an 
improvement  in  health  state  (Drummond  et  al.,  2000).  Intangible  value  is  created  when 
patients  or  caregivers  place  some  importance  on  establishing  a  definitive  diagnosis, 
independent  of  any  effect  the  information  has  on  treatment  and  clinical  outcomes. 
Intangible  value  is  created  by  the  reduced  uncertainty  from  obtaining  information  about 
underlying  causes  of  symptoms,  or  future  prognosis  of  an  illness  (Hirth  et  al.,  2000).  This 
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is  interesting  to  note  given  the  preponderance  of  early  dropouts  within  the  first  month 
of  mental  health  treatment  (Miller  et  al.,  1997).  Perhaps  the  intangible  value  created  from 
mental  health  care  is  satisfied  independent  of  treatment,  solely  through  diagnostic  and 
assessment.  Other  value  may  be  created  for  the  family  and  community  as  well  from  the 
reduced  uncertainty  and  reassurance  that  the  patient  is  receiving  appropriate  health  care. 

Rebecca  Blank  (2000)  points  out  one  novel  feature  that  uniquely  characterizes  mental 
health  care  is  the  reliance  on  “caring”  behaviors.  Providers  of  mental  health  care  devote 
extensive  treatment  time  to  interactions  that  communicate  a  sense  of  trust  and  caring  to 
the  patient  (listening,  empathy),  and  this  is  a  critical  component  of  the  patient-provider 
relationship.  Patients  may  associate  caring  behaviors  by  the  provider  with  higher  quality 
services,  but  from  a  cost  standpoint,  caring  is  a  very  inefficient  practice  that  creates 
higher  than  necessary  labor  costs  for  the  minimal  provision  of  services,  and  does  not  in 
itself  produce  tangible  benefits  or  necessarily  lead  to  better  health  outcomes.  It  may 
however,  improve  compliance  by  keeping  the  patient  interested  in  treatment.  The 
provider-patient  relationship  is,  in  fact,  one  of  a  strong  predictors  of  satisfaction  with 
care  (Moller-Leimkuhler  et  al.,  2002). 

Economic  research 

Although  it  benefits  from  strong  theoretical  support  treatment  noncompliance  is  a 
relatively  young  research  topic  in  economics  (Hughes  et  al.,  2001).  As  indicated  earlier 
most  of  the  research  that  has  taken  place  has  occurred  through  randomized  clinical  drug 
trials  and  through  economic  evaluations  comparing  the  costs  and  benefits  of  competing 
drug  therapies.  In  the  latter,  the  effects  of  noncompliance  are  evaluated  post  hoc 
through  a  sensitivity  analysis,  in  which  sides  of  the  both  cost  and  benefit  equation  are 
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varied  for  differing  assumptions  about  noncompliance  rates  (Drummond  et  al.,  2000)  In 
drug  trials,  compliance  may  be  evaluated  as  an  outcome  of  interest  (Cleemput  et  al., 
2002).  Some  researchers  argue  that  the  controlled  conditions  of  clinical  trials  are  ideal  for 
examining  noncompliance  since  all  potential  reasons  for  leaving  the  experiment,  other 
than  differences  in  the  treatment  themselves,  have  been  neutralized  by  randomization 
(Phillipson  &  Hedges,  1998). 

Patients  have  preferences  that  are  revealed  through  economic  choices,  and  choosing 
not  to  comply  is  such  a  choice  (Phillipson  &  Hedges,  1998;  Ellickson  et  al.,  1999).  These 
economic  choices  are  thought  to  have  high  information  value  and,  if  accurately  captured, 
could  be  helpful  in  reducing  noncompliance.  For  example,  several  recent  outcomes 
studies  have  been  conducted  which  link  patient  preferences  to  the  frequency  and  severity 
of  key  symptoms,  adverse  effects  of  treatment,  and  the  impact  of  disease  and  treatment 
on  daily  life  (Revicki,  Leidy,  Brennan-Diemer,  Sorenson,  &  Togias,  1998;  Ryan,  1999; 
Markson  et  al.,  2001;  McKenzie,  Cairns,  &  Osman,  2001).  These  studies  suggest 
methods  which  have  potential  for  enriching  our  knowledge  of  the  relationship  between 
patient  preferences  for  treatment  and  treatment  compliance.  Obtaining  patient 
preferences  for  care  however,  is  complex,  and  these  studies  are  very  new  to  the  research. 
More  commonly,  the  health  care  industry  deemphasizes  preferences  for  care  in  favor  of 
patient  satisfaction.  Thus  far,  the  notion  of  patient  satisfaction  has  been  somewhat 
implicit  in  the  discussion  of  patient  preferences,  but  a  closer  examination  is  needed  to 
clarify  the  relationship.  Although  commonly  used  in  health  care,  the  question  remains 
whether  current  measures  of  satisfaction  will  adequately  reveal  patient  valuations  which 
influence  decisions  about  compliance. 
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There  are  two  types  of  patient  preferences:  values  and  utilities,  and  the  distinction  is 
critical  to  informing  measurement.  As  discussed  utilities  are  preferences  under 
conditions  of  uncertainty  and  are  revealed  through  consumer  behavior,  such  as 
treatment  noncompliance  (Drummond  et  al.,  2000).  Revealed  preferences  can  be 
measured  using  willingness-to-pay,  time-trade  off,  standard  gamble  and  contingent 
valuation  exercises  which  simulate  uncertainty  by  introducing  probabilities  into  decision 
making.  For  example,  in  a  WTP  exercise,  the  point  of  indifference  between  two 
probabilistic  alternatives  is  used  to  obtain  minimum  amount  an  individual  would  pay 
rather  than  go  without  a  treatment.  There  are  several  problems  with  using  these 
exercises.  The  first  is  that  the  average  person  does  explicidy  consider  probabilities  which 
can  make  simulation  of  these  exercises  awkward.  Second,  individuals  are  accustomed  to 
not  paying  for  health  care  making  it  difficult  for  them  to  estimate  what  they  would  pay 
(Birks  &  Buurman,  1994).  Finally,  there  are  concerns  that  WTP  estimates  may  be 
constrained  by  an  individual’s  ability  to  pay,  and  valuations  are  therefore  dependent  on 
income  distributions.  (Drummond  et  al.,  2000).  However,  at  least  one  study  has  used 
WTP  as  a  proxy  for  patient  satisfaction  (Schiffner,  Brunnberg,  Hohenleutner,  Stolz,  & 
Landthaler,  2002)  finding  that  the  more  effective  the  treatment  modality,  the  more  the 
patient  would  be  willing  to  pay. 

Values  are  preferences  under  conditions  of  certainty  elicited  direcdy  from  consumers, 
and  therefore  called  stated  preferences.  Stated  preferences  are  commonly  measured 
through  scaling  methods,  such  as  rating,  visual  analogue  and  ratio  scales.  The  patient 
satisfaction  measure  is  a  rating  scale,  but  satisfaction  measures  suffer  from  several  known 
shortcomings.  Originating  in  business-not  economics,  satisfaction  measures  do  not  have 
a  sound  theoretical  base,  and  by  design,  they  ignore  the  notions  of  sacrifice  and 
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opportunity  costs  (Vick  &  Scott,  1998;  Scott  et  al.,  2002).  Although  they  elicit  patients’ 
experience  of  care,  they  do  not  direcdy  ask  about  preferences  for  treatment,  or  assess 
attributes  of  illness  that  are  most  important  to  patients-  which  the  literature  suggests  may 
be  at  the  heart  of  patient  decisions  not  to  comply.  Additionally,  satisfaction  data  can  be 
widely  misunderstood  and  misrepresented  as  indicative  of  superior  treatment  impact  or 
treatment  efficacy  (Lebow,  1982). 

Conclusion 

This  section  has  described  the  conceptual  framework  for  this  dissertation  research. 
Consumer  demand  for  health  care  is  derived  from  the  demand  for  health.  Utilization  is 
based  on  consumer  demand.  Demand  is  responsive  to  cost,  availability  of  substitutes  and 
the  utility  gained  from  care.  Utility  operates  within  the  constraints  of  household  budget 
and  is  composed  of  patient  preferences,  including  preferences  for  risk.  Health  care  has 
costs  and  benefits  beyond  what  the  individual  pays.  These  externalities  are  corrected 
through  government  intervention  to  subsidize  direct  costs  and  increase  supply.  This  is  an 
important  concept  in  mental  health  care,  where  more  care  may  be  available  than  would 
be  demanded  in  a  purely  market  environment,  but  in  a  market  environment,  the  care 
would  be  purchased  only  by  consumers  with  more  income. 

Individuals  with  a  mental  health  problem  are  often  disabled  and  have  low  incomes, 
and  these  are  exactly  the  persons  often  excluded  in  the  marketplace.  Although 
psychotherapy  has  treatment  benefits  for  any  individual  with  a  mental  health  problem, 
consumers  who  are  white,  well  educated  and  have  high  income  seem  to  have  a  greater 
demand  for  it.  Low  income  and  minority  consumers  may  be  more  interested  in  resolving 
basic  problems  of  food,  clothing,  shelter,  than  receiving  psychotherapy.  Therefore,  they 
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would  be  indifferent  to  being  priced  out  of  the  market  for  outpatient  care.  Outpatient 
mental  health  care  for  this  hard-to-reach  population  must  be  easily  affordable  and 
accessible.  Even  then,  as  mentioned  above,  if  the  product  is  wrong  then  demand  will 
remain  low.  Many  treatments  are  complex,  time  consuming  and  generate  other  indirect 
costs  for  the  patient.  Coupled  with  low  demand,  patients  may  find  the  cost  to  benefit 
ratio  is  too  large,  and  simply  drop  out  of  treatment. 

Economic  theory  provides  compelling  insight  into  the  persistent  problem  of 
noncompliance  with  mental  health  psychological  treatments.  The  role  of  patient 
preferences  in  treatment  noncompliance,  and  noncompliance  itself  as  utility  maximizing 
economic  behavior,  has  been  examined  in  pharmacoeconomic  research,  is  emerging  in 
health  care  research,  but  has  not  received  much  consideration  in  mental  health  care.  The 
notion  raises  questions  about  how  to  empirically  evaluate  the  relationship  between 
patient  preferences  and  the  resulting  economic  behavior.  Satisfaction  with  care  and 
treatment  noncompliance  are  less  than  ideal  proxies,  but  they  do  have  the  advantage  of 
highlighting  the  important  problem  of  treatment  noncompliance  in  outpatient  mental 
health  care,  and  testing  the  adequacy  of  patient  satisfaction  measurement  against  the 
highly  theoretical  utility  theory. 
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CHAPTER  3  Literature  review 
Gaps  in  Mental  Health  Research 

If  the  problem  of  treatment  noncompliance  dropouts  is  as  widespread  and 
problematic  as  suggested,  why  hasn’t  the  impact  been  previously  recognized  in  the 
research?  The  answer  lies  in  1)  the  evolution  of  mental  health  utilization  research  and 
interplay  over  the  years  between  research,  policy  and  payers;  2)  research  design  and 
sampling  issues  making  treatment  compliance  difficult  to  study;  and  3)  perceptions  by 
the  mental  health  industry  that  the  problem  is  systemic  and  intractable.  The  research 
community  first  became  interested  in  mental  health  utilization  after  the  1977  President’s 
Commission  on  Mental  Health  posed  serious  questions  about  the  extent  to  which 
Americans  with  mental  illness  were  being  underserved.  Deinstitutionalization  in  the 
1970’s  had  pushed  thousands  of  mentally  ill  persons  into  the  community,  and  from  this 
influx  of  people  needing  services  was  the  beginnings  of  the  community  mental  health 
system. 

The  1977  President’s  Commission  resulted  in  funding  for  the  Epidemiologic 
Catchment  Area  Study,  the  largest  mental  health  epidemiologic  survey  in  the  U.S.,  which 
sought  to  estimate  the  incidence  and  prevalence  of  mental  disorders  (Regier,  1994).  One 
powerful  finding  of  the  ECA  was  that  the  majority  of  persons  with  a  mental  disorder  do 
not  seek  care  (Shapiro  et  al,  1984).  Raising  public  concerns  about  access  to  services, 
policy  makers  and  the  mental  health  community  began  to  pressure  the  insurance  industry 
to  provide  outpatient  coverage  to  treat  individuals  with  mental  disorders.  As  tension 
grew,  it  generated  interest  among  researchers  regarding  the  effects  of  insurance 
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arrangements  on  mental  health  utilization.  Fueling  this  research  was  the  insurance 
industry  itself,  which  historically  believed  talk  therapies  were  not  efficacious  and 
necessitated  greater  price  controls  than  other  health  benefits  in  order  to  discourage 
discretionary  use. 

Two  questions  have  dominated  the  research  on  utilization  since  the  1980’s.  The  first 
is  whether  the  availability  of  health  insurance  increases  demand  for  mental  health  care 
(Horgan,  1983;  Keeler,  Manning,  &  Wells,  1988;  Zuvekas,  Banthin,  &  Selden,  2001). 
This  is  an  argument  about  parity  and  the  insurance  practice  of  providing  less  coverage 
for  mental  health  than  for  other  medical  services.  Several  large  studies  on  the  effects  of 
different  insurance  arrangements  on  demand  for  mental  health  were  carried  out  with  the 
most  well  known  being  the  Rand  Health  Insurance  Experiment  (HIE)  (Ware,  Manning, 
Duan,  &  Wells,  1984;  Diehl,  Price,  Williams,  &  Martin,  1986;  Watts,  Scheffler,  &  Jewell, 
1986;  Keeler  et  al.,  1988;  McGuire  &  Fairbank,  1988).  The  second  question  facing 
researchers  is  whether  reducing  incentives  to  discretionary  use  by  increasing  co-payments 
discourages  persons  with  greatest  need  from  seeking  care  (Ware  et  al.,  1984;  Taube  et  al., 
1986;  Landerman,  Burns,  Swartz,  Wagner,  &  George,  1994;  G.  E.  Simon,  Grothaus, 
Durham,  Vonkorff,  &  Pabiniak,  1996;  Goldman  &  Smith,  2001).  Coinciding  with  a  shift 
in  national  health  policy  and  concern  about  health  care  for  the  uninsured,  the  second 
question  inspired  another  generation  of  utilization  research  that  centers  around 
identifying  barriers  that  impede  access  to  care  (Horwitz,  1996;  Katz  et  al.,  1997;  R.  C. 
Kessler  et  al.,  1997;  Swartz  et  al.,  1998;  Hoff  et  al.,  1999;  Alpine  &  Mechanic,  2000). 


29 


Our  understanding  from  30  years  of  research  is  that  mental  health  care  is  price 
sensitive,  such  that  small  increases  in  copayments  result  in  large  decreases  in  the 
probability  of  use,  but  that  even  in  the  presence  of  zero  copayments  and  free  care,  many 
individuals  with  a  mental  health  diagnosis  will  still  not  seek  treatment.  Studies  have 
produced  consistent  information  about  who  will  and  will  not  use  the  mental  health  care 
system,  but  have  not  adequately  explored  the  question  of  why  (Landerman  et  al.,  1994; 
Swartz  et  al.,  1998).  What  is  evident  from  history  is  that  it  has  never  been  the  goal  of 
utilization  research  to  advance  our  knowledge  in  this  direction.  However,  our 
preoccupation  with  improving  access  for  those  who  will  not  seek  care,  and  reducing 
access  for  those  who  will  has  shifted  the  attention  further  away  from  weaknesses  in  the 
design  of  the  mental  health  care  system  itself.  To  predict  who  will  leave  and  who  will 
stay  in  therapy,  researchers  have  focused  on  patient  sociodemographic  characteristics 
and  disregarded  the  treatment  process  as  an  important  predictor  (Goldenberg,  2002). 
That  we  can  so  reliably  predict  utilization  based  on  demographic  factors  gives  reason  to 
wonder  if  the  population  for  which  the  mental  health  care  system  is  intended  is  the  same 
as  the  population  who  in  reality  use  the  mental  health  care  system.  It  is  evident  that  for 
older  persons,  African  Americans,  low  income  and  less  educated  persons,  the  current 
outpatient  mental  health  system  may  hold  very  little  appeal. 

A  second  reason  for  the  gap  in  mental  health  research  is  related  to  sampling  and 
design  issues  that  make  it  difficult  to  study  those  who  have  used,  but  who  terminate  early 
from  mental  health  care.  There  are  three  varieties  of  data  collection  commonly  used  in 
utilization  research:  1)  nationally  representative  samples  from  epidemiological  studies 
such  as  the  ECA  (Landerman  et  al.,  1994;  Swartz  et  al.,  1998),  National  Comorbidity 
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Survey  (NCS)(R.  C.  Kessler  et  al.,  1994;  Katz  et  aL,  1997;  R.  C.  Kessler,  Olfson,  & 
Berglund,  1998;  R.  C.  Kessler  et  al.,  1999),  National  Medical  Expenditure  Survey 
(NMES)  (Olfson  &  Pincus,  1994b;  Frieman  &  Zuvekas,  2000)  and  the  Medical 
Expenditure  Panel  Survey  (MEPS)  (Olfson  et  al.,  2002);  2)  less  generalizable,  but  often 
large  samples  of  clinical  populations  obtained  from  insurance  records  and  patient 
databases  (Hu,  Snowden,  Jerrell,  &  Nguyen,  1991;  Flaskerud  &  Hu,  1992;  Takeuchi  et  al., 
1995;  G.  E.  Simon  et  al.,  1996;  Hoff  et  al.,  1999);  and  primary  data  produced  from 
studies  designed  to  investigate  treatment  dropouts  (Pekarik,  1992;  Wierzbicki  &  Pekarik, 
1993;  I.  M.  Anderson  &  Tomenson,  1995;  Wang  et  al.,  2000). 

There  are  strengths  and  weaknesses  to  research  from  each  data  source.  For  example, 
large  surveys  can  produce  nationally  representative  estimates  of  mental  health 
expenditures  and  utilization,  and  can  be  used  to  estimate  demand  for  mental  health  care 
(health  seeking  behavior),  but  this  tells  us  little  about  changes  in  demand  that  occur 
among  persons  who  have  had  an  initial  contact  (continuity).  Large  surveys  provide  equal 
access  to  both  clinical  (in  treatment)  and  non  clinical  (no  longer  in  treatment) 
populations,  but  knowing  how  many  entered  care  and  how  long  they  stayed,  reveals  tittle 
about  the  conditions  under  which  they  left.  Researchers  must  rely  on  ‘level  of  use’  as  an 
imperfect  proxy  to  distinguish  those  who  terminate  care  as  dropouts  from  those 
terminate  care  as  completers  (Lebow,  1982;  Pekarik,  1992).  More  specifically,  those  who 
improve  spontaneously  with  consultation  from  those  who  receive  treatment  (Olfson  & 
Pincus,  1994b;  Bemdt  et  al.,  1998);  and  among  those  who  received  treatment  those  who 
completed  and  those  who  dropped  out  early  (Whynes,  1993;  Berndt  et  al.,  2000). 
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With  the  second  variety  of  research  there  is  the  opposite  problem.  Database  records 
can  be  used  to  discern  treatment  completers  from  dropouts,  but  researchers  do  not 
exploit  the  data  to  investigate  of  the  problem  or  its  implications  for  the  cost 
effectiveness  of  outpatient  treatments.  Data  from  patient  records  provide  better  exit  data 
on  patients,  but  almost  universally  the  patients  themselves  are  censored  once  they 
terminate,  excluding  them  from  further  study  and  biasing  research  findings  toward 
completers.  Researchers  have  recognized  this  weakness,  but  complain  of  grave  difficulty 
contacting  patients  once  they  have  terminated  (Pekarik,  1992).  Finally,  studies  using 
clinical  databases  do  not  give  adequate  consideration-  and  few  give  any  consideration-  to 
the  treatment  process  and  differential  preferences  for  treatment  as  a  possible  predictor 
of  treatment  dropout.  There  are  exceptions,  such  as  a  study  by  Goldenberg  (2002)  where 
researchers  differentiated  medication  from  non-medication  visits  as  predictors  of 
treatment  dropout.  Similarly,  in  a  study  by  Takeuchi,  et.al.,  (1995)  researchers  found 
treatment  setting  as  a  predictor  of  number  of  visits  to  be  significant  with  fewer  minority 
patients  in  ethnicity-specific  treatment  settings  terminated  treatment  early. 

The  third  variety  of  research  comes  from  observational  studies  of  clinical 
populations.  These  involve  some  combination  of  patient  survey  and  retrospective  chart 
review  in  which  researchers  can  tailor  the  research  design  to  collecting  the  right  variables 
and  countering  nonresponse  of  dropouts  (Cuffe,  Waller,  Cuccaro,  Pumariega,  & 
Garrison,  1995;  Grunebaum  et  al.,  1996).  For  example,  Pekarik  (1992)  sampled  100 
treatment  dropouts  specifically  investigating  reasons  for  terminating  care.  Halweg  & 
Klann(1997)  evaluated  attrition  in  495  marital  therapy  cases.  Kendall  &  Sugarman  (1997) 
evaluated  predictors  of  attrition  in  an  anxiety  disorder  clinic.  Wang,  et  al.  (2000) 


32 


conducted  a  cross-sectional  mail  survey  of  3,516  respondents  from  14  patients  advocacy 
groups  in  11  countries  asking  whether  patients  initiated  and  adhered  to  mental  health 
treatments  recommended  to  them.  Centorrino  et  al.  (2001)  in  a  study  of  patients  with 
missed  appointments  found  treatment  type  to  be  significant  predictor  of  noncompliance 
with  medication  patients  having  higher  rates  of  missed  appointments  than  psychotherapy 
patients.  Mueller  &  Pekarik  (2000)  conducted  a  prospective  study  examining  the  accuracy 
of  client  and  clinician  predictors  of  dropout. 

Well  designed  clinic-based  observational  studies  on  noncompliance  are  infrequent, 
and  do  not  produce  generalizable  findings  because  of  small  sample  sizes  and  lack  of 
random  selection.  Randomized  clinical  trials  that  use  treatment  dropout  as  an  outcome 
and  compare  differences  in  the  effectiveness  of  treatments  are  more  desirable,  but  are 
rarely  used  in  the  study  of  psychological  treatments  (Miller  et  al.,  1997).  One  exception  is 
a  randomized  controlled  trial  by  Hamilton,  Round,  &  Sharp  (1999)  comparing  rates  of 
missed  appointments  among  outpatients  who  were  sent  appointment  reminder  letters. 
Killapsy  et  al.  (2000)  conducted  a  prospective  controlled  study  at  a  London  outpatient 
psychiatric  clinic,  but  only  patient  factors  were  considered  as  predictors  of  missed 
appointments.  Randomized  clinical  trials  occur  more  frequently  in  mental  health  drug 
treatment  studies.  For  example,  in  a  recent  study  by  Tai-Seale,  Croghan,  &  Obenchein 
(2000),  researchers  use  noncompliance  rates  as  a  measure  of  treatment  effectiveness 
between  tricyclic  antidepressants  and  selective  serotonin  reuptake  inhibitors. 


Patterns  of  care  in  mental  health 
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A  major  concern  with  effectively  addressing  the  problem  of  treatment  dropouts  is 
that  the  mental  health  community  may  be  uninformed  about  the  extent  of  the  problem, 
or  may  interpret  noncompliance  as  part  of  the  burden  of  illness.  Researchers  have 
questioned  the  ability  of  mental  health  patients  to  rationally  evaluate  decisions  about 
compliance  because  of  impairments  in  cognitive  and  emotional  functioning  (Hoff  et  al., 
1999).  Pescosolido  &  Boyer  (1999)  state  “the  very  nature  of  mental  illness  calls  into 
question  the  individual’s  ability  to  engage  in  the  complicated  cognitive  processes  on 
which  some  [utilization]  theories  rely”  (p.  404).  Researchers  have  also  questioned 
whether  patients  with  mental  illness  are  able  to  objectively  formulate  evaluations 
regarding  satisfaction  with  care  (Hoff  et  al.,  1999).  Marshall  et  al  (1996)  question  the 
assertion  that  dissatisfaction  may  be  a  manifestation  of  psychological  distress  or  general 
life  dissatisfaction.  They  found  that  psychological  well  being  accounts  for  as  little  as  10- 
15  percent  of  the  variability  in  general  satisfaction  ratings. 

Failure  to  recognize  the  nature  and  extent  of  treatment  noncompliance  may  be  due  to 
asymmetric  information  qualities  inherent  in  the  provider-patient  relationship.  According 
to  a  1992  quote  by  Hurley,  Birch  and  Eyles  . .  while  the  health  care  provider  possesses 
better  knowledge  regarding  the  expected  effectiveness  of  health  care  in  improving  health 
status,  the  individual  knows  best  how  improvements  in  health  status  affect  his  or  her 
well  being.  Technical  knowledge  resides  in  one  party  while  preferences  reside  in  the 
other”  (Charles  et  al.,  1997)  (p.  683).  Patients  do  not  choose-but  are  expected  to  comply 
with-treatments  they  have  been  selected  into  by  their  providers.  A  recent  study  by 
Dwight-Johnson,  Unutzer,  Sherbourne,  Tang,  &  Wells  (2001)  found  in  a  sample  of 
depressed  patients  wanting  treatment,  many  did  not  get  the  treatment  they  preferred. 
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Providers  draw  upon  their  own  stock  of  knowledge  in  prescribing  treatments  and  do  not 
necessarily  act  neutrally  in  these  practices  (Donovan,  1995;  Lee,  1995;  Ellickson  et  al, 
1999).  Such  were  the  findings  in  a  study  comparing  patient  preferences  for  psychotropic 
medication  and  specialty  mental  health  care.  Frieman  &  Zuvekas  (2000)  found  providers 
suggest  treatments  in  part  on  the  basis  of  their  perceptions  of  patient  characteristics. 

An  even  simpler  explanation,  supporting  the  notion  that  the  professional  community 
may  be  unaware  of  the  extent  of  the  problem,  is  suggested  by  Howard  et  al.  (1989).  They 
describe  a  phenomenon  called  the  clinician's  illusion  which  occurs  because,  although  the 
majority  of  patients  attend  relatively  few  sessions,  the  majority  of  provider  time  is  spent 
with  a  small  number  of  long  term  patients.  This  creates  an  unrepresentative  impression 
on  the  provider  about  the  population.  The  authors  found  support  for  this  theory  in  a 
study  of  405  long  term  psychotherapy  patients.  Sixty-eight  percent  of  patients  attended 
fewer  than  27  sessions,  but  accounted  for  only  23.3%  of  the  total  sessions.  Thirty-two 
percent  attended  more  than  26  sessions,  and  accounted  for  77  percent  of  the  total 
number  of  sessions  used.  In  their  research  using  230  private  practice  patients,  Mueller  & 
Pekarik  (2000)  found  therapists  to  be  poor  predictors  of  duration  of  care  expecting  a  far 
greater  number  of  sessions  than  actually  occurred.  These  finding  were  similarly  reported 
by  Swain-Campbell,  Surgenor,  &  Snell  (2001)  in  research  on  eating  disorder  patients.  In  a 
study  by  Olfson  &  Pincus  (1994a),  researchers  used  NMES  data  to  identify  patterns  of 
utilization  among  psychotherapy  users.  With  number  of  visits  divided  into  very  short 
term:  1-2  sessions,  short  term:  3-10  sessions,  intermediate:  11-20  sessions  and  long  term: 
more  than  20  sessions,  they  found  that  long  term  psychotherapy  users  accounted  for 
only  15.9  percent  of  psychotherapy  users,  but  62.9  percent  of  psychotherapy 
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expenditures.  A  more  recent  study  by  Goldenberg  (2002)  of  2,889  patients  who  either 
completed  or  dropped  out  of  treatment,  found  that  one  quarter  of  clients  received 
between  one  and  four  sessions,  half  received  between  five  and  26  sessions  and  25 
percent  received  more  than  26  sessions. 

The  literature  suggests  almost  50  percent  of  people  making  a  mental  health  visits 
make  five  or  fewer  visits  (Howard  et  al.,  1989).  However,  clinical  practice  guidelines  for 
treatment  of  acute  phase  depression  do  not  indicate  any  demonstrated  effectiveness  in 
fewer  than  six  visits  (Bemdt  et  al,  1998).  Wang  et  al.  (2000)  reports  fewer  than  half  of 
patients  have  been  found  to  continue  treatment  for  adequate  duration.  Kendall  and 
Sugarman  (1997)  report  23  percent  of  therapy  clients  dropped  out  after  the  first  session 
and  almost  70  percent  terminate  before  the  10th  session.  Garfield  (1994)  reports  from 
investigations  conducted  over  several  decades  in  diverse  rltniral  settings,  that  the 
majority  of  clinics  lost  half  of  their  clients  before  the  eighth  sessions,  with  a  range  of 
three  to  13  and  a  cluster  around  six.  Reports  of  the  average  number  of  visits  ranges  from 
13  visits  reported  by  Howard  (1989)  to  more  recent  estimates  of  8.7  visits  reported  by 
Olfson  et  al.  (2002).  The  few  studies  that  have  gone  beyond  number  of  visits  to  compare 
dropouts  and  completers  have  reported  that  outcome  is  the  key  difference.  Goldenberg 
(2002)  found  treatment  outcome  to  be  the  strongest  correlate  in  number  of  visits,  with 
completers  showing  greater  improvement  and  dropouts  citing  lack  of  improvement  as 
the  reason  for  dropping  out.  Wang  et  al.  (2000)  found  lifetime  dropout  rates  due  to  side 
effects  and  treatment  ineffectiveness  to  be  44  percent.  Noncompliance  rates  do  not 
appear  to  be  constant  over  the  course  of  treatment.  Miller  et  al.  (1997)  reports 
compliance  during  the  first  month  is  the  best  predictor  of  long  term  compliance.  The 
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greatest  proportion  of  patients  discontinue  therapy  early  after  treatment  initiation  rather 
than  in  subsequent  months  or  years  (Hughes  et  al.,  2001).  Lee  (1995)  explains  this 
process:  Patients  go  to  the  doctor  to  gain  information  about  the  pattern  and  quantity  of 
medical  care  needed  for  health  improvement.  Crossing  the  doctor's  threshold  gives  the 
patient  a  chance  to  obtain  objective  information  about  the  care  needed,  but  the  health 
outcome  is  still  conditional  on  the  patient’s  health  status  and  the  patient’s  perception  of 
the  effectiveness  of  medical  care. 

Underlying  much  of  the  research  is  the  implication  that  person-centered  barriers, 
both  exogenous  i.e.,  insurance,  income,  and  endogenous  i.e.,  race,  age,  perceived  need, 
are  the  best  predictors  of  utilization  for  those  who  under-use  and  overuse  the  mental 
health  system  (Howard,  1996;  Killapsy  et  al.,  2000).  Although  it  creates  a  situation  where 
the  patient’s  response  to  the  mental  health  care  system  is  seen  as  outside  the 
responsibility  of  the  professional  community,  patient  sociodemographic  characteristics, 
such  as  race/ethnicity,  age,  gender,  income  and  education,  can  alos  provide  the  most 
compelling  evidence  for  the  influence  of  patient  preferences  on  low  rates  of  treatment 
and  treatment  noncompliance.  This  section  highlights  patient  factors  that  have  been 
associated  with  level  of  use,  treatment  noncompliance,  and  patient  satisfaction  in  mental 
health  care. 

Patient  characteristics 

Ethnicity 

The  strongest  evidence  of  differential  preferences  for  mental  health  care  is  by  ethnic 
group  where,  although  prevalence  of  mental  disorders  vary,  rates  of  outpatient  use  are 
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consistently  low  compared  to  whites  (Hu  et  ah,  1991;  Flaskerud  &  Hu,  1992;  Wierzbicki 
&  Pekarik,  1993;  Grunebaum  et  al.,  1996;  Swartz  et  ah,  1998;  L.  Cooper-Patrick  et  ah, 
1999),  and  this  pattern  has  not  changed  over  time  (Zuvekas,  2001).  This  is  particularly 
evident  with  African  Americans,  who  are  thought  to  have  rates  of  disorder  similar  to  that 
of  whites,  be  disproportionately  represented  among  the  seriously  mentally  ill,  but  have 
rates  of  outpatient  use  far  below  that  of  whites,  even  in  the  presence  of  insurance 
(Swartz  et  ah,  1998;  Alpine  &  Mechanic,  2000).  Research  reveals  African  Americans  tend 
to  have  greater  substitutions  for  care  such  as  family,  social  support  and  spirituality 
(Cuffe  et  ah,  1995),  and  as  a  group  endorse  greater  concerns  about  stigma  of  mental 
health  care  (Cuffe  et  ah,  1995;  Millett,  Sullivan,  Scwebal,  &  Myers,  1996;  L.  Cooper- 
Patrick  et  ah,  1997).  Stigma  is  a  not  often  considered  intangible  cost  of  mental  health 
care. 

Horwitz  (1996)  reports  ethnicity  is  strongly  related  to  both  entry  and  retention  in 
mental  health  treatment.  In  a  longitudinal  study  of  psychiatric  disorders,  Cuffe  et  ah 
(1995)  examined  race  as  a  factor  in  the  referrals  of  478  adolescents.  They  found 
Caucasian  patients  twice  as  likely  to  stay  in  treatment  past  two  visits  than  African 
American  patients.  Zuvekas  (2001)  reports  although  whites  are  still  more  likely  to  use 
MH/SA  services,  rates  of  treatment  between  1987-1996  grew  in  same  proportions  for 
whites,  African  Americans  and  Hispanics.  Hu  et  ah  (1991)  report  from  a  large  study  of 
the  public  mental  health  system  in  two  California  counties,  that  African  Americans  used 
more  emergency  and  less  outpatient  care.  Olfson  &  Pincus  (1994b)  found  that  nine  out 
of  ten  psychotherapy  users  were  white.  In  a  meta-analysis  of  125  psychotherapy  studies 
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Wierzbicki  &  Pekarik  (1993)  found  African  Americans  and  other  minorities  to  have  an 
increased  risk  of  dropping  out  of  psychotherapy. 

Frieman  &  Zuvekas  (2000)  in  a  study  comparing  patient  preferences  for  specialty 
mental  health  care  and  psychotropic  medications  found  that,  relative  to  whites,  other 
ethnic  groups  were  less  likely  to  have  either  type  of  treatment,  but  being  black  decreased 
the  probability  more-especially  for  receiving  specialty  care.  Zuvekas  (2001)  also  found 
African  Americans  were  also  less  likely  to  use  psychotropic  medication.  Hoff  et  al. 
(1999)  found  Caucasian  patients  having  higher  patient  satisfaction  ratings  compared  to 
African  American  patients.  Wells,  Klap,  Koike  and  Sherbourne  (2001)  have  found  both 
low  rates  of  treatment  and  low  satisfaction  among  Hispanics.  In  a  1995  study  of  5,000 
African  Americans,  Mexican  and  Asian  Americans  in  treatment,  Takeuchi  et  al.  (1995) 
found  that  all  ethnic  groups  in  ethnicity-specific  programs  were  more  likely  to  return 
after  one  visit  and  had  longer  lengths  of  treatment. 

Gender  and  age 

Research  has  linked  gender  and  age  to  patterns  of  mental  health  utilization.  Females 
are  more  likely  to  use  outpatient  mental  health  care  (H organ,  1983;  K.  Wells,  Manning, 
Duan,  Newhouse,  &  Ware,  1986;  Landerman  et  al.,  1994;  Olfson  &  Pincus,  1994b; 
Horwitz,  1996;  Hendryx,  2001;  Zuvekas,  2001).  Frieman  &  Zuvekas  (2000)  report 
probability  of  females  receiving  both  specialty  care  and  psychotropic  drugs  to  be  1.5  and 
2.5  percentage  points  respectively  higher  than  for  males.  Swartz  et  al.(1998)  report  the 
odds  of  use  in  the  public  sector  is  higher  for  males.  From  the  ECA  study  males  were 
more  likely  to  visit  a  specialist  (Shapiro  et  al.,  1984).  Few  studies  have  been  done  linking 
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age  and  gender  specifically  to  treatment  noncompliance  or  satisfaction  with  care 
(Wierzbicki  &  Pekarik,  1993;  Hoff  et  al.,  1999;  Fontana  &  Rosenheck,  2001).  However, 
Wang  et  al.(2000)  found  that  being  male  increased  the  odds  of  treatment  compliance. 
Frieman  &  Zuvekas  (2000)  found  significant  differences  between  males  and  females  in 
their  preferences  for  specialty  care  and  medication.  Hoff  et  al  (1999)  reports  males  to  be 
more  satisfied  with  care,  but  the  sample  was  drawn  from  a  predominately  male 
population. 

Using  NCS  data  to  estimate  lifetime  and  12  month  prevalence  of  mental  disorders  R. 
Kessler  et  al.  (1994)  found  the  highest  prevalence  of  disorder  occur  in  25-34  year  olds 
with  prevalence  declining  thereafter.  Horwitz  (1996)  in  his  literature  review  on  help¬ 
seeking  concluded  that  age  was  strongly  related  to  mental  health  use  with  the  elderly 
being  particularly  reluctant  to  enter  care.  The  odds  of  public  sector  use  is  nearly  six 
times  less  likely  among  oldest  versus  youngest  cohort  (Swartz  et  al.,  1998).  Low  rates  of 
treatment  among  the  elderly  may  be  due  to  lower  rates  of  disorder,  higher  rates  of 
institutionalization  and  poor  mental  health  coverage  among  Medicare  benefits  (Swartz  et 
al.,  1998;  Zuvekas,  2001).  Being  over  the  age  of  65,  decreases  the  likelihood  of  receiving 
long  term  psychotherapy  while  the  35-49  age  group  are  more  likely  to  use  psychotherapy 
(Olfson  &  Pincus,  1994b).  Being  older  is  associated  with  greater  satisfaction  with  mental 
health  care  (Hoff  et  al.,  1999;  Fontana  &  Rosenheck,  2001). 

Education  and  income 

Higher  levels  of  education  are  associated  with  increased  mental  health  use  and 
adherence  to  treatment.  Greater  education  is  associated  with  increased  probability  of 
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seeing  a  mental  health  specialist  (Ware  et  al.,  1984;  Frieman  &  Zuvekas,  2000) 
andreceiving  medication  (Frieman  &  Zuvekas,  2000).  Diehr  et  al.  (1986)  and  Mueller  & 
Pekarik  (2000)  reported  higher  education  is  associated  with  number  of  visits.  Olfson  & 
Pincus  (1994a)  found  having  less  than  12  years  of  education  decreased  the  likelihood  of 
receiving  long  term  psychotherapy.  Less  education  is  also  associated  with  higher  rates  of 
dropout  (Wierzbicki  &  Pekarik,  1993)  and  more  education  associated  with  greater 
adherence  to  treatment  (Centorrino  et  al.,  2001).  Education  effects  appear  to  be  much 
greater  in  predicting  mental  health  use  than  income  effects  and,  in  some  cases,  even 
insurance  effects  (Swartz  et  al.,  1998).  This  is  consistent  with  the  health  capital  model 
which  purports  individuals  with  more  education  may  have  a  higher  value  for  health  and 
use  more  health  care  inputs  (Grossman,  1972). 

Taube  et  al.,  (1986)  reports  income,  education  and  insurance  coverage  interact  to 
produce  different  demand  elasticities  across  income  groups.  For  example,  those  with  less 
income  are  also  less  likely  to  have  insurance  required  to  access  mental  health  care  (Shi, 
2001).  Monheit,  Vistnes,  &  Zuvekas  (2001)  disagree  finding  individuals  in  the  lowest 
income  groups  have  greater  stability  in  their  public  insurance  coverage  than  any  other 
income  group  except  high  income.  Individuals  who  are  near  poor,  without  Medicaid,  and 
whose  household  income  is  1 50  percent  of  the  poverty  level  are  at  higher  risk  than  other 
income  groups  for  having  mental  health  service  needs  unmet  (Klerman,  Olfson,  Leon,  & 
Weissman,  1992).  Hendryx  (2001)  found  higher  income  associated  with  a  greater  number 
of  mental  health  visits.  Others  have  found  no  direct  relationship  between  income  and 
utilization  (Horwitz,  1996).  Olfson  and  Pincus  (1994a)  found  income  only  weakly  related 
to  psychotherapy  use  with  poor  or  near-poor  receiving  long  term  psychotherapy  in 
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proportion  to  their  numbers  in  the  general  population.  Flaskerud  &  Hu  (1992)  report 
patients  in  lower  SES  receive  less  treatment,  but  earlier  studies  showed  lower  SES  to  be  a 
significant  predictor  of  at  least  one  visit  (Diehr  et  al.,  1986).  Wierzbicki  and  Pekarik 
(1993)  report  drop  out  rates  increase  for  lower  income  groups.  Hoff  et  al.(1999)  found 
high  income  associated  with  greater  satisfaction. 

Insurance 

The  amount  of  insurance  coverage  is  a  major  factor  in  the  use  of  mental  health 
services  (Goodman,  1989;  K.  B.  Wells  et  al.,  1990;  Landerman  et  al.,  1994).  Horgan 
(1983),  Keeler  (1988)  and  Taube  (1986)  report  cost  sharing  influences  the  demand  for 
mental  health  care,  but  the  decision  to  use  services  is  affected  differently  than  level  of 
use.  For  example,  a  study  by  Simon,  Grothaus,  Durham,  Vonkorff  and  Pabiniak  (1996) 
estimated  that  instituting  a  $20  co-payment  can  have  a  16  percent  reduction  in  the 
likelihood  of  use,  and  $30  co-payment  can  decrease  visits  by  nine  percent.  Recent 
changes  in  the  payment  structure  of  insurance  has  also  dramatically  influenced  use. 
According  to  Zuvekas  (2001),  by  the  1990's  three  fourths  of  insured  Americans  had  their 
MH/SA  services  managed  under  managed  carve-out  plans.  McGuire  and  Fairbank 
(1988)  the  HMO  effect  on  mental  health  care  is  primarily  on  visits  per  user  rather  than 
on  the  number  of  users.  Rosenthal  (2000)  estimates  under  capitated  plans,  fixed 
payments  to  providers  for  cases  has  reduced  the  number  of  visits  by  20-25  percent. 

There  have  been  concerns  about  whether  higher  insurance  co-payments  produce 
more  barriers  to  mental  health  care  for  those  with  the  greatest  need.  Using  findings  from 
the  HIE,  Ware  et  al.  (1984)  rejected  insurance  as  a  factor  for  those  with  greatest  need 
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not  receiving  care.  They  found  among  persons  under  the  free  care  plan,  the  most 
psychologically  distressed  still  had  only  a  one  in  five  chance  of  receiving  outpatient 
mental  health  care.  Horwitz  (1996)  also  found  that  out-of-pocket  costs  of  mental  health 
care  have  a  strong  impact  on  use  regardless  of  clinical  need.  However,  Landerman  et  al. 
(1994)  found  the  effect  of  insurance  coverage  on  the  probability  of  a  visit  was  stronger 
among  those  with  a  diagnosis.  There  are  no  studies  which  estimate  the  effects  of  co¬ 
payments  on  the  decision  to  terminate  care  prematurely,  although  there  likely  to  be  a 
connection  given  that  the  strongest  insurance  effects  seem  to  be  on  number  of  visits. 
There  is  no  data  on  insurance  coverage  and  satisfaction  with  care,  although  it  is 
reasonable  to  expect  patients  with  more  generous  insurance  benefits  may  be  more 
satisfied  with  their  health  care.  Insurance  could  also  follow  other  socioeconomic  factors 
that  have  been  linked  to  satisfaction,  such  as  income  and  age. 

Health  Status 

Outpatient  mental  health  use  is  common  among  individuals  in  poor  health  and  with 
functional  impairments  due  to  chronic  disease  (Olfson  &  Pincus,  1994b).  Diehr  et  al., 
(1986)  found  poor  health  and  lower  SES  was  correlated  with  having  at  least  one  mental 
health  visit,  but  better  health  and  higher  SES  was  correlated  with  a  higher  quantity  of  use 
for  those  who  had  some  use.  Olfson  &  Pincus  (1994a)  found  poor  or  fair  health  status 
was  associated  with  long  term  psychotherapy  use.  Horgan  (1983)  reports  neither 
perceived  health  status  nor  disability  days  were  significantly  related  to  number  of  visits. 
Health  status  and  mental  health  status  may  have  a  different  effect  on  compliance  than 
patient  sociodemographic  factors.  Patient  preferences  may  play  a  greater  role  on 
decisions  to  comply  with  care,  but  health  status  could  be  a  more  important  factor  in  the 
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initial  decision  to  seek  care  (Lee,  1995).  In  a  study  of  114  rheumatology  patients  Hall, 
Milburn,  Roter,  &  Daltroy  (1998)  found  less  healthy  patients  tend  to  be  less  satisfied 
with  their  care.  Hoff  et  al.  (1999)  also  found  persons  with  lower  health  ratings  had  lower 
satisfaction  ratings.  Marshall  et  al.  (1996)  found  no  significant  links  between  general 
satisfaction  with  care  and  physical  health  status. 

Diagnosis/Mental  health  status 

Research  findings  suggest  mental  health  status  is  a  major  predictor  in  both  probability 
and  intensity  of  outpatient  mental  health  treatment  (Ware  et  al.,  1984;  Olfson  &  Pincus, 
1994a;  Hoff  et  al.,  1999;  R.  C.  Kessler  et  al.,  1999;  Frieman  &  Zuvekas,  2000;  Killapsy  et 
al.,  2000;  Centorrino  et  al.,  2001).  The  presence  of  a  psychiatric  disorder  seems  to  be  the 
strongest  predictor  of  probability  of  use  (Katz  et  al.,  1997;  Swartz  et  al.,  1998;  L.  Cooper- 
Patrick  et  al.,  1999),  and  level  of  use  (L.  G.  Kessler,  Steinwachs,  &  Hankin,  1980; 
Flaskerud  &  Hu,  1992;  Landerman  et  al.,  1994).  Landerman  et  al.  (1994)  estimates  from 
ECA  data  that  those  with  a  diagnosis  are  six  times  as  likely  as  others  to  obtain  care. 
Flaskerud  &  Hu  (1992)  found  diagnosis  to  be  significantly  related  to  number  of  sessions. 

Goldenberg  (2002)  reports  a  weak  association  between  diagnosis  and  number  of  non¬ 
medication  visits.  Alpine  and  Mechanic  (2000)  report  the  number  of  outpatient  visits 
varies  little  across  disorders,  even  for  persons  with  serious  mental  illness.  Katz  et  al. 
(1997)  found  the  prevalence  of  perceived  need  for  mental  health  care  is  consistently 
higher  in  the  U.S.  than  in  Canada,  but  R.  C.  Kessler  et  al.(1997)  found  differences  in  the 
probability  of  obtaining  care  in  the  U.S.  compared  to  Canada  was  confined  to  those  with 
less  severe  mental  illness.  Some  researchers  have  found  patients  who  are  more  acutely  ill 
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at  the  time  they  seek  treatment  are  more  compliant  than  those  chronically  ill  or  in  partial 
remission  (Centorrino  et  al.,  2001).  Killapsy  et  al.  (2000)  reports  that  patients  who  miss 
appointments  are  more  unwell  and  more  socially  impaired.  Grunebaum  et  al.(1996) 
suggests  patients  with  mild  distress  and  those  with  significant  resistance  to  seeing  a 
psychiatrist  were  more  likely  to  miss  appointments.  Hoff  et  al.  (1999)  found  patients 
with  a  mental  health  diagnosis  to  have  lower  satisfaction  ratings  for  both  mental  health 
and  heath  care. 

While  there  is  a  dearth  of  research  that  investigates  the  role  of  patient  characteristics 
in  determining  utilization,  there  is  an  absence  of  research  that  investigates  the  role  of 
patient  preferences  in  determining  treatment  dropouts.  Patient  preferences  for  treatment 
have  been  studied  in  the  context  of  patient  satisfaction  and  patient  satisfaction  has  been 
viewed  as  a  desirable  outcome  of  treatment  (Ware,  1997).  There  are  several  problems 
with  the  conceptualization  of  patient  satisfaction  and  an  entire  section  is  devoted  to  a 
review  of  measurement  issues.  However,  relative  to  gaps  in  the  research,  it  is  important 
to  note  that  of  the  studies  conducted  patient  characteristics  such  as  health  status  and 
diagnosis  are  most  often  employed  to  predict  patient  satisfaction  (Marshall  et  al.,  1996; 
Halweg  &  Klann,  1997;  Hah  et  al.,  1998;  Lambert  et  al.,  1998;  Hoff  et  al.,  1999).  In  a 
study  of  3,600  veterans  being  treated  for  PTSD,  Fontana  &  Rosenheck  (2001)  found 
patient  characteristics  outweighed  treatment  structure  and  treatment  effectiveness  in 
determining  satisfaction,  but  treatment  structure  was  defined  as  waiting  time  to 
admission  and  length  of  stay. 
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The  relationship  between  satisfaction  with  care  and  treatment  outcomes  has  been 
investigated  in  several  correlational  studies  (Baronet  &  Gerber,  1997;  Lambert  et  al., 
1998;  Pekarik  &  Guidry,  1999;  Markson  et  al.,  2001).  Only  Pekarik  (1992),  Brannan 
(1996)  and  Mueller  &  Pekarik  (2000)  have  examined  satisfaction  as  a  predictor  of 
treatment  duration.  Several  interesting  non-mental  health  studies  have  examined 
satisfaction  in  terms  of  patient  preferences.  Schiffner  et  al.  (2002)  used  utility  measures, 
willingness-to-pay  and  time  trade-off  exercises,  as  proxies  for  satisfaction  with  treatment 
modality.  Scott  et  al.  (2002)  evaluated  patient  preferences  for  out-of-hours  care  using  a 
series  of  discrete  choice  experiments.  These  did  not  examine  effects  of  preferences  on 
duration  of  care.  Few  mental  health  studies  to  date  have  used  national  probability  survey 
data  to  examine  patient  satisfaction.  The  MEPS  devotes  an  entire  interview  supplement 
to  access,  quality  and  satisfaction  with  care. 

Taken  together  the  literature  suggests  some  patients  are  reluctant  to  utilize  the  mental 
health  care  system.  This  flies  in  the  face  of  what  is  known  about  the  prevalence  of 
mental  disorders  and  perceived  need  for  care  among  individuals  with  a  mental  disorder. 
Patients  who  use  and  do  not  use  the  mental  health  system  stand  out  by 
sociodemographic  characteristics  such  as  age,  gender,  race,  insurance,  education.  This 
research  seeks  support  for  the  hypothesis  that  although  patients  may  face  personal  and 
environmental  constraints  in  their  decisions  about  health  care,  the  widespread 
persistence  and  prevalence  of  high  rates  of  treatment  noncompliance  suggests  patients 
may  be  responding  to  factors  within  the  care  system  itself  in  their  decision  to  follow 
through  with  treatment.  Instruments  which  measure  patient  satisfaction  with  care  should 
offer  some  glimmer  of  information  about  patient  valuations  of  these  system  factors.  The 
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following  section  provides  a  review  of  the  patient  satisfaction  literature  to  evaluate 
current  trends  in  patient  satisfaction  measurement  and  explore  issues  of  measurement 
design,  validity  and  reliability. 
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Understanding  Patient  Satisfaction 

Several  concerns  exist  with  regard  to  patient  satisfaction  measures.  A  critical  review 
of  the  literature  reveals  two  gaps:  wrong  population  and  wrong  measure.  Responses  are 
universally  skewed  toward  high  satisfaction  (Pollack  &  Wachter,  1975;  Carr-Hill,  1992; 
Plante  et  al.,  1998).  According  to  Fornell  &  Johnson  (1993)  taken  alone  the  negative 
skewness  means  little  since  it  could  imply  that  people  buy  what  they  like,  and  if  they 
don’t  like  what  they  bought  they  will  not  buy  it  again.  They  suggest  this  could  be 
interpreted  as  an  indicator  of  economic  health  since  it  means  consumers  have  choices. 
However,  if  the  diminishing  rates  of  follow-up  visits  can  be  equated  to  repurchase 
behavior  then  it  lends  greater  support  to  the  argument  that  many  patients  do  not  like 
what  they  bought,  and  that  current  study  design  may  only  be  reaching  the  patients  who 
did.  The  population  of  treatment  dropouts-  those  whose  experiences  and  preferences  we 
most  need  to  understand  -  have  proven  to  be  elusive.  This  section  provides  a 
comprehensive  review  of  the  satisfaction  literature  with  two  goals  in  mind.  First,  two 
explore  the  complexity  of  patient  satisfaction  measurement  and  second  to  describe  the 
content  and  structure  of  existing  empirically  tested  instruments. 

The  two  most  common  validity  problems  in  satisfaction  measurement  are  selection 
bias  and  acquiescence.  These  problems  result  in  high  undifferentiated  levels  of 
satisfaction  that  produce  no  systematic  variance  and  make  it  difficult  to  distinguish 
between  levels  of  satisfaction  (Hudak  &  Wright,  2000).  Selection  bias,  as  illustrated  in  the 
paragraph  above,  refers  to  having  derived  data  from  a  non-representative  sample,  and 
according  to  Lebow  (1982),  Pekarik  (1992)  and  Marshall  et  al.  (1996)  is  one  of  the  most 
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serious  threats.  In  outpatient  mental  health  settings  where  treatments  extend  over  time, 
dissatisfied  patients  are  more  likely  to  withdraw  from  treatment  early,  resulting  in  an 
over-representative  sample  of  patients  who  on  average  have  better  outcomes  and  are 
more  satisfied. 

Inadequate  research  design  can  contribute  to  selection  bias  (Carr-Hill,  1992).  The 
most  frequently  occurring  design  flaw  is  timing  of  data  collection.  In  a  study  by  Pekarik 
&  Guidry  (1999)  researchers  called  for  instruments  to  be  administered  at  four  and  eight 
months  after  intake,  which  is  well  beyond  the  point  when  most  patients  drop  out. 
Brannan  et  al.  (1996)  administered  study  instruments  only  to  clients  receiving  services 
longer  than  one  month.  In  a  study  on  satisfaction  and  treatment  outcomes  Plante  et  al. 
(1998)  report  that  many  patients  terminated  before  three  month  and  six  month  measures 
could  be  administered.  In  research  by  Halweg  &  Klann  (1997)  where  satisfaction 
measures  were  to  be  administered  three  and  six  months,  49  percent  of  the  sample  had 
terminated  by  the  six  month  data  collection  point.  How  much  of  a  sampling  frame  is  lost 
to  dropout  is  difficult  to  estimate  because  researchers  do  not  uniformly  report  attrition 
rates  in  research  findings.  There  are  exceptions  such  as  a  study  by  Pekarik  &  Guidry 
(1999)  where  researchers  reported  that  from  a  sample  of  160  private  practice  patients, 
data  was  collected  on  only  93  patients  due  to  attrition.  Addressing  systematic  differences 
in  the  populations  that  may  have  influenced  findings  is  even  more  important,  but  rarely 
occurs  in  satisfaction  research  (Nabatd,  Shea,  McBride,  Gavin,  &  Bauer,  1998). 

A  second  validity  problem  with  satisfaction  measures  is  acquiescence,  or  the 
tendency  of  respondents  to  agree  with  statements  of  opinion  regardless  of  content. 
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(Hudak,  2000).  In  his  early  research  on  patient  satisfaction  measurement,  Ware  (1997) 
found  10  percent  of  clients  were  totally  acquiescent  in  surveys  and  another  40  percent 
were  at  least  somewhat  acquiescent.  Single  item  satisfaction  measures  and  measures 
containing  all  favorably  or  unfavorably  worded  questions  are  most  biased  toward 
acquiescence  (Hudak  &  Wright,  2000).  Avis  et  al.  (1997)  found  that  asking  patients  about 
their  overall  satisfaction  often  results  in  positive  replies,  whereas  specific  questions  about 
the  actual  processes  of  care  elicit  a  critical  response.  Markson  et  al.  (2001)  in  their 
research  on  satisfaction  with  asthma  treatments  found  by  framing  the  questions  in  terms 
of  dissatisfaction:  “Are  you  dissatisfied  with  any  aspect  of  your  asthma  treatment?”  they 
could  detect  far  more  variance  in  patient  responses.  In  fact,  in  this  cross-sectional  study 
of  5,181  asthma  patients,  30  percent  indicated  dissatisfaction  with  their  treatment. 

The  construct  validity  of  patient  satisfaction  measures  has  been  particularly  difficult 
to  establish.  Researchers  report  that  items  on  patient  satisfaction  questionnaires  often 
represent  important  dimensions  from  the  researchers’  perspective  but  may  lead  to 
inflated  reports  of  high  satisfaction  from  the  patients’  perspective  (Baronet  &  Gerber, 
1997;  L.  Cooper-Patrick  et  al.,  1997).  Avis  et  al.  (1997)  explain  that  the  use  of  open- 
ended  questions  and  semi-structured  interviews  in  satisfaction  research  prevents  eliciting 
answers  that  only  reflect  providers’  assumptions  about  what  is  important  to  patients. 
Since  the  content  of  measures  discriminating  satisfaction  from  dissatisfaction  has  not 
been  established,  and  existing  constructs  have  been  questioned,  some  researchers  have 
selected  a  qualitative  research  design.  For  example,  Cooper-Patrick  et  al.  (1997) 
conducted  focus  groups  with  African  American  and  white  patients  to  explore  differences 
in  help  seeking,  treatment  preferences  and  perceived  barriers  to  mental  health  care. 
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Baronet  &  Gerber  (1997)  used  a  combination  of  survey  and  open-ended  questions  on 
their  research  with  community  crisis  center  clients.  Another  reason  for  choosing  a 
qualitative  component  is  to  evaluate  the  concurrent  validity  of  standardized  instruments. 
Williams  et  al.  (1998)  used  unstructured  interviews  to  validate  the  Client  Satisfaction 
Questionnaire  (CSQ-18b)  finding  expressions  of  satisfaction  on  the  CSQ-18b  hid  a 
variety  of  negative  expressions  that  were  present  during  the  interviews. 

Lebow  (1982),  Carr-Hill  (1992),  Pekarik  and  Guidry  (1999)  and  Markson  et  al.  (2001) 
suggest  the  problem  is  that  patients  may  have  difficulty  expressing  dissatisfaction.  The 
infrequency  of  ill  health  gives  patients  limited  experience  from  which  to  evaluate  quality 
of  treatment  and  outcomes  (Johnson,  Nader,  &  Fornell,  1996).  Uncertainty,  risk  and 
asymmetric  information  surrounding  the  diagnosis,  treatment  and  prognosis  of  medical 
problems  makes  evaluating  satisfaction  especially  complex  for  the  patient.  Contrasting 
with  other  service  industries,  the  patient  is  an  integral  part  of  the  ‘production  process’  in 
health  care.  They  may  have  difficulty  separating  their  contribution  to  a  health  care 
intervention  from  that  of  the  provider  (Arrow,  1963;  Fornell  &  Johnson,  1993;  Bearden, 
Malhoyta,  &  Uscategui,  1998),  and  may  even  blame  themselves  for  failure  to  achieve 
hoped  for  outcomes  (Avis  et  al.,  1997).  Health  care  is  a  continuously  provided  service,  of 
which  a  distinctive  feature  is  that,  upon  entering  into  a  relationship  with  the  provider,  the 
patient  consumes  or  uses  the  service  either  continuously  or  intermittently  over  an 
extended  period.  The  extent  to  which  patient  satisfaction  with  care  that  occurred  over  an 
extended  period  converges  with  satisfaction  from  a  specific  medical  encounter  is  an 
important  distinction  having  important  implications  for  measurement,  but  which  has  not 
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been  clearly  evaluated  in  health  care  research  (Marshall,  Hays,  Sherboume,  &  Wells, 
1993;  Rao  et  al.,  2000). 

The  role  of  expectations  in  how  patients  formulate  satisfaction  evaluations  has  been 
researched  extensively  in  marketing  science,  and  no  fewer  than  ten  major  conceptual 
models  have  been  produced  and  tested  internationally.  Suffice  it  to  say  expectations  play 
an  important  role  in  the  evaluation  of  any  product  or  service,  and  in  health  care  have 
been  linked  both  to  satisfaction  (Williams  et  al,  1998)  and  number  of  visits  (Mueller  & 
Pekarik,  2000;  Rao  et  al.,  2000).  The  exact  nature  of  the  relationship  is  less  clear.  For 
example  Williams  et  al.  (1998)  finds  little  consistent  empirical  evidence  that  fulfillment  of 
expectations  necessarily  results  in  satisfaction.  According  to  Cuffe  et  al.  (1995) 
‘expectation’  is  a  latent  construct  that  is  contextually  derived.  It  must  be  defined  within  a 
given  context  before  it  is  suitable  for  measurement.  Rust,  Inman,  Jia,  &  Zahorik  (1999) 
explain  that  expectations  are  derived  from  another  latent  construct:  uncertainty,  and 
uncertainty  depends  on  the  amount  of  experience  a  consumer  has  had  with  a  product  or 
service.  Nonetheless  there  are  practical  concerns  for  the  effects  of  general  pessimism, 
optimism  on  expectations  and  responses  to  questions  about  satisfaction  with  care.  This 
is  a  particularly  important  for  the  mental  health  population  in  which  some  believe 
negativity,  pessimism  and  irrational  thinking  invalidates  satisfaction  measures  altogether. 

Patient  satisfaction  studies  in  health  care  have  sought  to  identify  illness  factors  that 
predict  satisfaction  with  care,  such  as  chronic  illness,  mental  health  status  and  diagnosis 
(Marshall  et  al.,  1996;  Atkinson  &  Caldwell,  1997;  Hall  et  al.,  1998;  Hoff  et  al.,  1999; 
Healey,  Mirandola,  Ammaddeo,  Bonizzato,  &  Tanzella,  2000).  Researchers  have 
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conceptualized  satisfaction  as  itself  a  treatment  outcome  (Ankuta  &  Abeles,  1993; 
Marshall  et  al.,  1993;  Pekarik  &  Wolff,  1996;  Ware,  1997;  Lambert  et  al.,  1998;  Plante  et 
al.,  1998;  Pekarik  &  Guidry,  1999),  but  there  is  an  inherent  danger.  By  conceptualizing 
satisfaction  as  an  outcome,  researchers  have  created  satisfaction  measures  that  measure 
outcomes  not  satisfaction,  such  as  a  study  by  Ankuta  &  Abeles  (1993).  Their  measure  of 
satisfaction  consisted  of  four  questions  from  the  Strupp  Post  Therapy  Client 
Questionnaire.  Three  questions  were  related  to  benefit  from  therapy  and  amount  of 
change:  ‘benefit  from  therapy’,  ‘perceived  amount  of  change’  and  ‘perceived  current 
functioning’.  ‘Satisfaction  with  therapy’  was  the  only  satisfaction  question.  It  is  no 
surprise  that  researchers  found  a  relationship  between  clinically  significant  change  and 
greater  satisfaction.  Researchers  have  reported  satisfaction  measures  do  a  very  poor  job 
of  measuring  outcomes  finding  that  satisfaction  and  treatment  outcomes  are  distinctly 
separate  constructs  that  do  not  even  co-vary  (Ankuta  &  Abeles,  1993;  Landerman  et  ah, 
1994;  Lambert  et  ah,  1998).  Barker  &  Orrell  (1999)  report  that,  although  patient 
satisfaction  has  been  employed  as  both  a  predictor  and  an  outcome  in  the  research,  its 
value  resides  as  a  predictor  variable  in  its  influence  on  outcomes  such  as  compliance. 
Despite  its  theoretical  support,  far  less  research  has  been  conducted  using  satisfaction  as 
a  predictor. 

A  link  between  patient  satisfaction  and  compliance  has  been  suggested  in  several 
studies  (Lebow,  1982;  Cramer  &  Rosenheck,  1998;  Barker  &  Orrell,  1999;  Rao  et  ah, 
2000).  Pekarik  (1992)  examined  satisfaction  among  treatment  dropouts  reporting  that 
dissatisfaction  was  greatest  among  unimproved  dropouts.  Nabati  et  ah  (1998)  tested  the 
relationship  between  low  satisfaction  scores  and  treatment  dropout  finding  no 
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association,  but  the  study  focus  was  on  psychometric  properties  of  the  satisfaction 
measure  with  satisfaction  hypothesi2ed  as  a  unidimensional  construct.  Sherboume  et  al. 
(1992)  used  data  from  the  Medical  Outcomes  Study  to  assess  the  relationship  between 
adherence  and  satisfaction  among  chronically  ill  patients  finding  satisfaction  with 
financial  and  interpersonal  aspects  of  care  were  better  predictors  of  adherence  than 
satisfaction  with  technical  aspects  of  care.  In  a  study  of  eating  disorder  patients,  Swain- 
Campbell  et  al.  (2001)  found  treatment  type  to  be  the  largest  source  of  dissatisfaction 
and  incongruencies  between  clinician  and  patient  expectations  to  be  an  important  cause 
of  premature  treatment  dropout.  Satisfaction  with  care  has  been  used  in  many  mental 
health  studies  where  satisfaction  itself  is  not  the  primary  variable  of  interest.  Measured 
using  a  single  global  question,  these  reveal  little  variation  in  satisfaction  (Pollack  & 
Wachter,  1975;  Ankuta  &  Abeles,  1993;  Pekarik  &  Wolff,  1996;  Mueller  &  Pekarik, 
2000). 

Satisfaction  studies  in  mental  health  have  focused  on  the  development  and  validation 
of  instruments  to  identify  the  best  structure  (Marshall  et  al.,  1993;  Brannan  et  al.,  1996; 
Avis  et  al.,  1997;  McKinley,  Manku-Scott,  Hastings,  French,  &  Baker,  1997;  Nabati  et  al., 
1998;  Barker  &  Orrell,  1999;  MoUer-Leimkuhler  et  al.,  2002)  and  content  (Marshall  et  al, 
1993;  Landerman  et  al.,  1994;  Brannan  et  al.,  1996;  Ware,  1997;  Lambert  et  al,  1998).  Of 
14  studies  reviewed  13  were  devoted  to  instrument  development.  What  follows  is  a 
review  of  major  studies  that  address  the  content  validity  and  performance  of  previously 
used  measures.  The  purpose  of  providing  this  review  is  to  guide  measure  selection  for 
this  study.  There  are  two  major  objectives:  The  first  is  to  identify  the  content  of 
satisfaction  measures  which  have  generated  the  most  empirical  support.  The  second  is  to 
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determine  whether  to  combine  multiple  satisfaction  questions  into  a  single 
unidimensional  measure. 


Unidimensional 

Unidimensional  measures  hypothesize  patient  satisfaction  as  a  single  construct.  There 
have  been  two  basics  strategies  in  the  research:  survey  questions  about  global 
perceptions  of  care,  and  questions  about  multiple  domain-specific  aspects  of  care  that 
are  then  summarized  to  represent  overall  satisfaction  (Marshall  et  al.,  1993).  Global 
satisfaction  questions  have  been  found  to  be  less  valid  than  questions  about  distinct 
dimensions  of  care  (Carr-Hill,  1992;  Pekarik  &  Wolff,  1996;  Baronet  &  Gerber,  1997; 
Meredith,  Orlando,  Humphrey,  Camp,  &  Sherboume,  2001).  However,  Pekarik  (1992) 
used  a  single  global  question  “In  general,  how  satisfied  are  you  with  the  services 
received?”  measured  on  a  four  point  scale  to  lend  support  for  the  hypothesis  that 
treatment  dropouts  are  not  a  homogeneous  group.  Using  this  unidimensional  measure 
researchers  found  client  satisfaction  differed  significantly  among  three  groups  of 
dropouts.  Although  high  in  face  validity,  directness  and  ease  of  administration,  global 
questions  increase  the  likelihood  of  high  undifferentiated  patient  responses,  and  even  if 
they  produce  enough  variability  to  warrant  additional  statistical  testing,  they  do  not 
reveal  what  aspects  of  the  care  resulted  in  satisfaction-  and  more  importantly 
dissatisfaction. 

Combining  multidimensional  items  into  a  single  index  of  satisfaction  has  been 
criticized.  The  decision  is  often  based  on  how  hypothesized  dimensions  of  satisfaction 
factor  together.  When  items  are  submitted  to  a  principal  component  analysis,  there  is 
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sometimes  only  one  dominant  factor  that  accounts  for  the  greatest  amount  of  variance  in 
patient  response.  Researchers  may  decide  to  retain  only  these  items  combining  them  to 
produce  a  single  dimension.  (Marshall  et  al.,  1993)  found  aggregated  indexes  to  be  more 
reliable  than  the  individual  items  that  compose  them  and  less  sensitive  to  method  error. 
Hosmer  and  Lemeshow  (2000)  suggest  that  a  collection  of  variables,  each  of  which  is 
only  weakly  associated  with  the  outcome  may  together  be  an  important  predictor.  Other 
researchers  point  out  multidimensional  measures  cannot  necessarily  be  combined  into 
one  overall  index  of  satisfaction  without  compromising  reliability  (Lebow,  1982; 
Marshall  et  al.,  1993;  Hoff  et  al,  1999).  In  summary,  not  enough  is  known  is  known 
about  the  validity  of  satisfaction  measures  in  discriminating  among  underlying  constructs 
to  warrant  combining  them  into  a  unidimensional  measure. 

Multidimensional 

Multidimensional  measures  that  tap  into  patient  valuations  of  diverse  treatment 
characteristics  and  clinical  phenomena,  such  as  provider  patient  relationship,  technical 
competence  and  accessibility  seem  to  perform  better  than  global  satisfaction  measures. 
(Marshall  et  al.,  1993;  Pekarik  &  Guidry,  1999).  Multidimensional  measures  have  been 
heavily  influenced  by  the  health  care  quality  movement  originating  with  the  1966  paper 
by  Donabedian  defining  patient  satisfaction  as  the  ultimate  validation  of  quality,  and 
quality  as  the  degree  of  effectiveness  of  care  in  achieving  and  producing  health.  Ware 
and  Davies,  who  were  among  the  first  to  define  satisfaction  as  a  multidimensional 
concept,  suggested  the  five  most  important  dimensions  of  satisfaction  are  quality  of  care, 
accessibility/convenience,  finance,  physical  environment  and  availability  (Barker  & 
Orrell,  1999).  Commonly,  measures  of  patient  satisfaction  are  limited  to  dimensions  of 
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health  care  that  are  easy  to  improve  such  as  waiting  times  and  friendliness  of  personnel, 
but  a  study  by  Markham,  Diamond,  &  Hermansen  (1999)  found  that  waiting  time,  for 
example,  was  least  important  to  patients  accounting  for  only  15  percent  of  the  observed 
variance  in  patient  preferences. 

Two  patient  satisfaction  instruments,  the  Patient  Satisfaction  Questionnaire 
developed  by  Ware,  Snyder  and  Wright  in  1976  (Marshall  et  al.,  1993)  and  the  Client 
Satisfaction  Questionnaire  (CSQ-18)  developed  by  Larsen,  Atkisson,  Hargreaves  and 
Ngyen  and  later  shortened  to  eight  items  (CSQ-8)  by  Nguyen,  Attkinsson  and  Stegner 
(Plante  et  al,  1998)  have  dominated  the  satisfaction  literature  in  mental  health.  This 
section  discusses  the  content  and  structure  of  these  and  similar  measures.  The  basic 
approach  has  been  to  survey  several  hypothesized  dimensions  using  separate  items  or 
subscales  with  or  without  an  additional  question  or  subscale  for  global  satisfaction. 
Reliability  is  established  through  test-rest  where  scores  are  compared  on  two  occasions 
to  measure  stability  over  time,  and  through  inter-item  reliability  using  Cronbach’s 
coefficient  alpha  or  the  coefficient  of  determination  to  provide  information  on  the 
proportion  of  total  variance  the  items  have  in  common  (Kerlinger  &  Lee,  2000).  Inter¬ 
item  reliability  reveals  the  correlations  of  items  within  each  subscale  with  most 
researchers  accepting  only  items  that  have  correlations  of  0.50  or  better  indicating  at 
least  a  moderate  correlation.  Factor  analysis  is  then  used  to  as  a  construct  validity  tool  to 
ensure  subscales  are  measuring  distinctly  different  underlying  dimensions  or  latent 
factors,  and  to  identify  those  that  together  account  for  the  most  variance  (Kerlinger  & 
Lee,  2000).  What  comprises  the  correct  dimensions  is  contextual  with  a  wide  range  of 
specificity  with  a  tendency  toward  nonspecific  content  as  mentioned  earlier. 
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Baronet  &  Gerber  (1997)  used  the  CSQ-18  to  conduct  a  mixed  methods  study  of 
satisfaction  with  a  sample  of  99  clients  from  community  crisis  center.  The  correlation 
coefficient  from  test-retest  procedures  for  total  satisfaction  scores  on  the  CSQ-18  was 
0.90.  Chi-square  test  revealed  statistically  significant  differences  between  satisfied  and 
dissatisfied  clients.  Findings  indicated  that  40  percent  of  total  variance  in  their  model  was 
explained  by  client  evaluations  of  the  worker,  such  as  listening,  attitude  and  caring 
behaviors.  The  CSQ18b  was  used  by  Williams  et  al.  (1998)  in  another  mixed  methods 
study  comparing  the  CSQ18b  to  unstructured  patient  interviews.  They  found  that 
expressions  of  satisfaction  on  the  CSQ-18b  hid  a  variety  of  reported  negative 
experiences  that  were  derived  from  unstructured  interviews.  Plante  et  al.  (1998) 
conducted  a  cross-sectional  study  of  115  6-17  year  olds  using  the  CSQ-8  to  evaluate  the 
relationship  between  treatment  outcome  and  client  satisfaction.  The  CSQ-8  was 
administered  with  outcome  measures  at  three  month  intervals.  Satisfaction  remained 
high  throughout  the  study  and  was  not  associated  with  outcome  measures,  age  or 
gender. 

Pekarik  &  Guidry  (1999)  conducted  a  study  comparing  satisfaction  with  outcomes 
measures  at  four  and  eight  months  using  the  Client  Satisfaction  Rating  (Pekarik,  1996). 
The  instrument  consisted  of  four  items  each  measured  on  a  five-point  scale:  ‘overall  how 
satisfied  are  you  with  the  services  received’;  ‘would  you  recommend  agency  to  others 
seeking  help’;  ‘if  you  sought  help  again,  would  you  return  to  agency’;  ‘how  would  you 
rate  your  therapist’.  They  found  intercorrelations  between  0.59  to  0.84  among 
satisfaction  items  and  satisfaction  ratings  at  four  months  highly  correlated  with 
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satisfaction  ratings  at  eight  months  (r=0.88).  Marshall  et  al.  (1996)  examined  the 
association  between  self-reported  health  status  and  satisfaction  with  health  care  using  the 
PSQ-III.  The  PSQ-III  is  a  50  item  questionnaire  with  subscales  for  general  satisfaction, 
interpersonal  manner,  communication,  technical  competence,  time  spent  with  doctor, 
financial  aspects,  accessibility  and  convenience,  with  each  individual  item  measured  on  a 
five-point  scale.  Subscales  were  constructed  by  averaging  across  items  within  each 
dimension.  Using  factor  analysis  on  the  17  composite  indexes,  researchers  found  both  a 
six-factor  and  a  higher  order  model  revealing  that  satisfaction  can  be  conceptualized 
hierarchically  through  an  overarching  general  domain  with  a  set  of  discrete  dimensions 
tapping  into  unique  aspects  of  satisfaction. 

Moller-Leimkuhler  et  al.  (2002)  found  global  satisfaction  to  be  highly  correlated  with 
discrete  questions  about  satisfaction.  Performing  a  correlation  analysis  between  24 
distinct  items  on  the  Munich  Patient  Satisfaction  Scale  (MPSS-24)  with  six  global 
satisfaction  questions  on  the  MPSS-72,  they  found  the  correlation  between  patient 
doctor  relationship,  cooperation  and  ward  atmosphere,  patient  evaluation  of  nurse 
behavior  and  global  satisfaction  to  be  0.71.  Using  just  the  24-item  multidimensional 
structure,  a  principal  component  analysis  with  varimax  rotation  was  performed  finding  a 
three-factor  solution  with  one  dominant  factor  accounting  for  37.8  percent  of  the  total 
variance.  The  dominant  factor  was  the  patient-doctor  relationship  i.e.,  communication, 
empathy,  efficacy  of  pharmacological  treatment.. 

The  doctor-patient  relationship  has  been  established  as  a  feature  of  health  care  that 
patients  do  feel  is  important  and  most  instruments  will  measure  this  in  some  fashion. 


59 


Meredith  et  al.  (2001)  conducted  research  evaluating  the  correlation  between 
interpersonal  aspects  of  the  doctor-patient  relationship  and  satisfaction  with  health  care. 
Their  satisfaction  measure  was  a  six-item  questionnaire  measuring:  explanation  of  health 
problems,  being  given  a  choice  about  treatment,  involvement  in  decisions  about  care, 
ease  of  getting  help  with  emotional  or  personal  problems,  global  satisfaction  with  overall 
health  care  and  global  satisfaction  with  health  care  for  personal  or  emotional  problems. 
The  Interpersonal  Patient  Provider  Rating  Scale  consisted  of:  patient  can  talk  about 
personal  things  with  the  provider,  patient  comfort  asking  questions  of  the  provider,  the 
provider  listens  to  the  patient,  patient  control  over  treatment  and  provider  understanding 
of  how  the  patient  is  feeling.  Researchers  compared  the  fit  of  one-factor  and  two-factor 
solutions  using  confirmatory  factor  analysis  and,  despite  the  similarity  of  content 
between  their  satisfaction  and  interpersonal  relationship  measures,  they  found  evidence 
for  two  distinct  constructs. 

McKinley  et  al.  (1997)  examined  the  reliability  and  validity  of  a  32-item 
multidimensional  satisfaction  scale  for  out-of-hours  care.  With  the  Eigenvalue  fixed  at 
one,  a  principal  components  analysis  identified  six  components:  communication  and 
management  (7  items  a  =  0.88);  doctor’s  attitude  (5  items  a  =0.87);  continuity  of  care  (4 
items  a  =0.69);  delay  until  visits  (3  items  a  =0.65);  access  (3  items  a  =0.61);  and  initial 
contact  person  (2  items  a=0.72).  Items  were  summed  and  expressed  as  percentage  of  the 
maximum  possible  score  for  that  component.  Each  subscale  tended  to  be  highly 
correlated  with  overall  satisfaction  with  care  which  itself  was  comprised  of  four  items 
and  yielded  a  Cronbach’s  alpha  coefficient  of  0.77  for  internal  consistency. 
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Brannan  et  al.  (1996)  examined  patient  satisfaction  over  continuum  of  care  as  part  of 
the  Fort  Bragg  Evaluation  Project  (FBEP).  Researchers  used  the  FBEP  Satisfaction  Scale 
which  is  a  29-item  questionnaire  measuring  five  latent  factors:  access  and  convenience  (5 
items),  child’s  treatment  process  and  relationship  with  therapist  (5  items),  patient  and 
family  services  (4  items),  and  global  satisfaction  (4  items).  Overall  internal  consistency 
for  the  ten  subscales  was  >0.70.  Confirmatory  factor  analysis  was  used  to  determine 
whether  content  areas  of  satisfaction  scales  represent  distinct  dimension  of  satisfaction. 
A  five-factor  model  was  hypothesized,  but  a  four-factor  provided  the  best  fit. 
Researchers  found  a  correlation  between  agreement  regarding  termination  and  greater 
satisfaction.  Lambert  et  al.(1998)  looked  at  the  correlation  between  satisfaction  with  care, 
symptom  change  and  perceived  improvement.  As  part  of  the  FBEP,  children  5-17 
receiving  treatment  and  their  parents  were  administered  questions  from  the  FBEP 
satisfaction  scales  and  service  specific  satisfaction  questions:  global  satisfaction, 
helpfulness  and  perceived  improvement.  Researchers  tested  five  models  hypothesizing 
different  combinations  of  the  three  using  confirmatory  factor  analysis.  They  found  no 
correlation  between  satisfaction  and  symptom  change. 

Hoff  et  al.  (1999)  conducted  a  study  evaluating  whether  patients  with  psychiatric 
diagnosis  were  less  satisfied  with  care  than  non-psychiatric  patients.  The  outcome 
measure,  satisfaction  with  care,  consisted  of  43  questions  related  to  coordination  of  care, 
open  sharing  of  information,  courtesy  of  staff,  emotional  support,  attention  to  patient 
preferences/involvement  in  decision-making,  quality  of  family  involvement,  physical 
comfort  and  attention  to  physical  needs  and  help  with  transition  out  of  hospital.  Inter¬ 
item  reliability,  if  tested,  was  not  reported.  Although  the  measure  had  the  potential  to 
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assess  several  dimensions  of  care  including  preferences,  it  was  converted  to  a 
unidimensional  measure.  Subscale  scores  were  averaged  together  and  weighted  for  the 
number  of  items  in  each  subscale.  From  their  linear  regression  model  they  found 
psychiatric  diagnosis  to  be  a  significant  predictor  of  satisfaction  with  care. 

Nabati  et  al.  (1998)  examined  the  psychometric  properties  of  the  12-item  Satisfaction 
Index  for  Mental  health  on  a  sample  of  163  psychiatric  patients  selected  at  random  from 
waiting  areas  in  Veterans’  Administration  healthcare  facilities.  They  used  principal 
components  analysis  with  only  components  having  eigenvalues  greater  than  1.0 
considered  significant.  They  found  one  major  and  one  minor  component  representing 
communication,  humaneness,  ‘improvement,  satisfaction,  confidence  that  together 
demonstrated  an  internal  consistency  of  0.90.  Researchers  concluded  satisfaction  is 
unidimensional  and  items  were  summed  into  a  single  score. 

Research  on  patient  satisfaction  using  measures  as  they  are  currently  designed  have 
contributed  minimally  to  our  understanding  of  satisfaction.  Even  characteristics  of  the 
provider-patient  relationship  which  measure  better  place  a  heavy  burden  on  providers 
and  perpetuate  the  continued  use  of  measures  that  only  reveal  high  satisfaction.  Current 
measures  seem  to  miss  the  mark  regarding  satisfaction  with  the  treatment  itself.  One 
approach  to  measuring  satisfaction  that  has  met  some  success  are  utility  measures.  These 
measures  such  as  willingness-to-pay,  time-trade  off  and  other  scenario-driven  exercises 
require  respondents  make  choices  based  on  ordered  selections  to  the  point  of 
indifference  between  alternatives.  Schiffner  et  al  (2002)  used  these  methods  to  assess 
satisfaction  with  port  wine  stain  patients.  Patients  provided  their  subjective  impression 
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of  degree  of  improvement  in  the  appearance  of  the  port  wine  stains  following  laser 
treatments.  Utility  questions  involving  the  patient’s  willingness-to-pay  for  an  imaginary 
therapy  to  relieve  their  disease  was  used  as  a  measure  of  satisfaction.  They  found  patients 
evaluating  their  care  as  good  were  willing  to  pay  more  for  the  treatment  than  patients 
evaluating  their  care  as  bad.  Patients  with  greater  quality  of  life  impairment  following 
surgery  were  willing  to  pay  more  than  patients  with  less  quality  of  life  impairment. 

In  a  study  of  300  randomly  selected  patients  Markham  et  al.  (1999)  used  conjoint 
analysis  to  investigate  the  relative  importance  of  factors  in  doctor-patient  interactions. 
Researchers  created  eight  different  scenarios  for  time  in  waiting  room,  medical  skill  of 
doctor,  time  to  get  referral,  amount  of  billing  problems  and  who  makes  care  decisions 
with  each  measured  on  a  five  point  scale.  Creating  a  factorial  design  of  every  possible 
combination,  they  tested  eight  of  the  scenarios.  They  found  perceived  skill  of  the 
physician’s  accounted  for  27  percent  of  the  variance  in  patient  preferences.  Scott  et  al. 
(2002)  conducted  a  stated  preference  discrete  choice  experiment  for  out-of-hours  care. 
From  48  possible  scenarios,  16  were  generated  using  a  fractional  factorial  design.  One 
scenario  was  treated  as  constant  to  which  the  other  scenarios  were  compared.  The 
fifteen  scenarios  remaining  were  split  into  two  questionnaires  and  administered  to  a 
sample  of  3,893  primary  care  patients.  They  found  the  most  important  attribute  was 
whether  the  doctor  seemed  to  listen. 

The  state  of  the  research  on  patient  satisfaction  measurement  can  be  characterized  by 
1)  reliance  on  qualitative  research  identifying  relevant  sources  of  dissatisfaction,  but  with 
little  integration  of  findings  into  follow-on  quantitative  research;  2)  lack  of  focus  and 
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consistency  in  instrumentation  and  more  importandy  pursuit  of  what  may  be  relatively 
unimportant  sources  of  dissatisfaction  from  the  patient’s  point  of  view;  and  3)  the 
absence  of  replication  studies  with  different  populations  and  instead,  the  creation  of  new 
instruments  with  every  study.  There  is  still  much  work  to  be  done  conceptually  in  the 
measurement  of  patient  satisfaction,  and  this  study  does  little  to  make  revolutionary 
improvements.  However,  it  does  fill  two  important  gaps.  First,  the  research  employs 
patient  satisfaction  as  a  predictor  variable  which  has  been  suggested  but  not  carried  out. 
Second,  it  is  one  of  the  first  studies  to  use  national  survey  data  to  evaluate  patient 
satisfaction,  and  the  first  study  to  use  national  data  to  examine  the  relationship  between 
patient  satisfaction  treatment  noncompliance. 
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CHAPTER  4  Data  and  measures 

The  literature  suggests  there  are  unexplained  differences  between  individuals  who 
terminate  after  the  first  few  visits  and  individuals  who  go  on  complete  outpatient  care. 
This  research  goes  one  step  further  to  suggest  those  differences  may  in  part  be  a 
function  of  patient  satisfaction.  It  is  hypothesized  that,  after  controlling  for  patient 
factors,  levels  of  patient  satisfaction  will  be  associated  with  treatment  noncompliance. 
This  section  describes  the  three  research  hypotheses  and  how  MEPS  data  was  re¬ 
constructed  to  support  each  of  hypothesis.  Second,  the  criteria  and  method  for  case 
selection  into  the  study  will  be  discussed  to  include  handling  of  missing  and  truncated 
data.  Third  an  explanation  of  variables  selection  is  provided  to  include  method  and 
testing  of  the  predictor  variables,  selection  of  sociodemographic  and  impairment 
variables,  and  construction  of  the  compliance  measure  for  the  depression  subgroup. 

Three  test  hypotheses  were  formulated  to  examine  the  relationship  (Figure  1).  The 
literature  suggests  the  first  few  visits  are  critical  times  in  decision-making  about 
subsequent  appointment  compliance.  The  first  analysis  concentrates  on  how  levels  of 
satisfaction  affect  the  probability  of  having  two,  three  and  four  visits.  The  emphasis  is  on 
how  the  probabilities  of  continuing  treatment  change  in  the  presence  of  high  and  low 
satisfaction.  To  maximize  the  chances  of  observing  an  association  between  satisfaction 
with  care  and  treatment  compliance  in  this  model,  probabilities  in  the  first  four  visits  are 
compared  with  probabilities  for  continued  cate  beyond  four  visits.  The  second  analysis 
broadens  the  focus  to  the  effects  of  patient  satisfaction  on  continuation  at  any  point  in 
care.  I  examine  differences  in  patient  satisfaction  across  the  entire  duration  of  care, 
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predicting  the  hazard  rate  will  be  significantly  different  between  high  and  low  levels  of 
satisfaction.  The  outcome  of  interest  is  time  to  dropout,  or  the  failure  rate  among  highly 
satisfied  compared  to  dissatisfied  respondents. 

Selecting  cases  where  treatment  ended  between  one  and  four  visits  as  a  proxy  for 
treatment  dropouts,  and  cases  where  treatment  ended  with  more  than  four  visits  as  a 
proxy  for  treatment  completion  is  imprecise.  A  third  analysis  will  performed  that 
employs  minimum  treatment  standards  to  better  simulate  dropouts  and  completers.  A 
cohort  of  respondents  with  a  depression  diagnosis  are  chosen  for  this  analysis  because  1) 
there  is  a  practice  guideline  clearly  stating  a  minimum  number  of  visits  for  efficacious 
treatment,  and  2)  depression  is  a  prevalent  disorder  and  comprises  the  largest  single 
diagnostic  subgroup  in  the  MEPS.  The  AHRQ  clinical  practice  guideline  for  treatment 
of  acute  stage  depression  will  be  used  (Agency  for  Health  Care  Policy  and  Research, 
1993).  In  this  scenario,  respondents  completing  fewer  than  six  visits  are  presumed  to 
have  terminated  care  before  a  minimum  level  of  treatment  has  been  attained  (Berndt  et 
al,  1998)  (Melfi,  Chawla,  Croghan,  &  Hanna,  1998).  Respondents  completing  six  or 
more  visits  are  presumed  to  have  attained  an  adequate  level  of  treatment.  Because  a 
respondent  could  have  fewer  than  six  visits,  but  taking  medication  and  thus  still  in 
treatment,  the  compliance  variable  will  account  for  medication.  Olfson  et  al.  (2002) 
estimate  as  many  as  79.4%  of  the  1997  MEPS  population  are  treated  with 
pharmacotherapy  making  the  remaining  sample  of  non-medication  respondents  for  this 
analysis  quite  small. 
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Hypothesis  1  There  will  be  a  relationship  between  satisfaction  with  care  and  the  probability  of 
second,  third  and  fourth  visits. 

Hypothesis  2  The  hazard  rate  for  attrition  from  mental  health  care  will  differ  for  respondents  with  low 
satisfaction  and  those  with  high  satisfaction. 

Hypothesis  3  In  a  depression  practice  guideline  derived  subgroup  analysis,  respondents  reporting 
higher  satisfaction  will  be  more  likely  to  have  six  or  more  visits  than  respondents  reporting  lower 
satisfaction,  IlllaH  .. 


Figure  1  Research  Hypotheses 

Data  for  this  study  is  drawn  from  1996-1998  and  1998-1999  Agency  for  Healthcare 
Research  and  Quality  (AHRQ)  Medical  Expenditure  Panel  Survey  (MEPS)  which 
provides  nationally  representative  estimates  of  U.S.  health  care  use  and  expenditures 
(Agency  for  Healthcare  Research  and  Quality,  2001).  The  choice  of  secondary  data  over 
primary  data  collection  was  driven  by  two  sampling  issues.  First,  only  a  small  portion  of 
the  U.S.  population  use  mental  health  care  each  year  which  necessitates  the  use  of  a  large 
data  set  to  achieve  an  adequate  sample.  Second,  although  clinical  settings  are  ideal  for 
observing  patterns  of  use  and  distinguishing  early  success  from  early  termination,  they 
are  an  inadequate  source  of  data  on  individuals  once  they  have  left  care.  Household 
based  studies  with  a  longitudinal  component  such  as  the  MEPS  ensures  the  availability 
of  respondent  data  independent  of  active  treatment.  Neither  of  these  requirements  could 
be  adequately  satisfied  in  a  primary  study  without  major  time  and  funding  commitments. 
The  choice  of  MEPS  over  other  national  health  care  datasets  was  motivated  by  the 
availability  of  key  research  variables:  satisfaction  with  care,  treatment  type,  diagnosis, 
medications,  impairment  and  sociodemographic  characteristics. 
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This  research  builds  on  previous  research  in  several  ways.  First,  it  includes 
empirically  tested  covariates  previously  shown  to  be  related  to  either  or  both  predictor 
and  outcome  variables.  Second,  it  employs  a  measure  of  patient  satisfaction  that  is 
consistent  with  the  content  and  structure  of  previously  used  measures.  Third,  it  employs 
number  of  visits  as  a  measure  of  appointment  compliance  with  outpatient  mental  health 
care.  The  research  improves  on  previous  research  in  several  ways.  First,  satisfaction  with 
care  is  employed  as  a  predictor  instead  of  as  an  outcome  of  care.  Second,  it  is  the  first 
research  to  model  the  relationship  between  satisfaction  with  care  and  treatment 
compliance  using  a  national  sample.  Third,  it  is  one  of  the  only  studies  on  satisfaction 
and  noncompliance  to  use  household,  versus  clinically  or  administrative  based  data. 

The  research  is  not  able  to  improve  on  the  research  in  three  major  ways.  First,  there 
is  no  way  to  distinguish  treatment  dropouts  from  completers  in  absolute  terms.  The 
formulation  of  three  hypotheses  that  tests  three  different  visit  thresholds  for 
noncompliance  is  expected  to  offset  this  weakness.  Second,  the  design  of  variables  in 
MEPS  lacks  specificity  to  study  treatment  preferences-where  predictors  of 
noncompliance  are  thought  to  truly  exist.  For  example,  the  variable  for  psychotherapy  is 
a  catch-all  for  every  kind  of  psychologically-based  treatment  from  individual  and  group 
psychotherapy,  marital  therapy  and  psycho-educational  interventions,  to  patient 
management  of  medication  regimens.  Patient  preferences  may  vary  greater  between  each 
of  these  treatment  types.  Third,  the  data  does  not  support  satisfaction  evaluations  based 
exclusively  on  experiences  with  mental  health  care.  Instead,  responses  to  satisfaction 
questions  are  based  on  overall  evaluations  of  health  care.  Importance  weights  must  be 
designed  to  minimize  the  influence  of  responses  from  individuals  with  proportionally 
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more  health  care  utilization  and  maximize  the  influence  of  responses  from  individuals 
with  proportionally  less  mental  health  care  utilization.  Fourth,  the  data  does  not  easily 
support  the  distinction  of  visits  to  multiple  providers  as  long  as  the  purpose  was  a 
mental  health  visit  nor  it  is  easy  to  distinguish  separate  episodes  of  care  that  may  occur 
closely  together. 

MEPS  is  a  stratified  multi-stage  area  probability  design  household  survey  with  an 
overlapping  panel  design  (J.  W.  Cohen,  1997).  Data  are  collected  over  a  30  month  period 
to  produce  two  calendar  years  of  health  care  utilization  and  expenditure  data  with  a  new 
panel  beginning  each  year.  Five  rounds  of  interviews  are  carried  out  at  three  to  four 
month  intervals,  with  each  interview  designed  to  obtain  data  on  health  care  that  occurred 
during  that  round.  This  research  uses  data  from  two  panels:  Panel  1  which  consists  of 
five  rounds  of  household  interviews  that  together  produce  data  for  1996-1997,  and  Panel 
3  which  consists  of  five  rounds  of  interviews  that  produce  data  for  1998-1999.  Panel  2 
for  1997-1998  was  not  an  option  because  the  Access  to  Care  supplement  where  patient 
satisfaction  questions  reside  is  not  available. 

Each  sampling  frame  for  the  MEPS  is  attained  from  the  previous  year’s  respondents 
to  the  National  Health  Interview  Survey  (NHIS)  conducted  by  the  National  Center 
Health  Statistics  (S.  B.  Cohen,  2000).  NHIS  is  a  stratified  multistage  area  probability 
design  with  oversampling  of  African  Americans  and  Hispanics  at  rates  of  2:1  and  1.5:1 
respectively.  The  full  NHIS  sample  each  year  consists  of  358  primary  sampling  units  and 
approximately  42,000  household  across  the  country.  The  sub-sample  drawn  for  the  1996 
MEPS  Panel  1  Rounds  1-5  were  selected  from  two  of  the  four  1995  NHIS  panels  and 
resulted  in  an  un- weigh  ted  sample  of  22,601  individuals.  The  sample  for  the  1998  MEPS 
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Panel  3  Rounds  1-5  were  drawn  from  the  1997  NHIS  for  a  sample  of  11,137  individuals. 
For  this  study,  only  respondents  with  at  least  one  mental  health  visit  are  selected  yielding 
a  sample  of  approximately  1,206  individuals  and  15,090  visits  for  1996-1997  with  an 
additional  557  individuals  and  7,854  visits  for  1998-1999.  Differences  in  sample  size  are 
due  to  the  exclusion  of  Panel  2  observations  in  1998. 

MEPS  is  a  component  based  survey.  The  Household  Component  (HC)  collects 
detailed  data  on  respondent  medical  and  demographic  characteristics:  medical 
conditions;  health  status;  use  of  health  care  services;  charges  and  payments;  access  and 
satisfaction  with  care;  health  insurance  coverage;  income  and  employment  (Agency  for 
Healthcare  Research  and  Quality,  2001).  The  HC  was  designed  to  produce  national  and 
regional  estimates  of  health  care  utilization,  expenditures,  sources  of  payment  and 
insurance  coverage  of  the  U.S.  civilian  non-institutionalized  population.  S.  B.  Cohen 
(1996)  reports  because  of  the  complex  design  of  the  MEPS,  sample  data  must  be 
appropriately  weighted  to  obtain  approximately  unbiased  national  estimates  for  the  U.S. 
civilian  non-institutionalized  population.  Sampling  weights  reflect  the  disproportionate 
sampling  that  results  from  oversampling  of  minority  population,  complete  nonresponse 
of  eligible  sample  units,  partial  response  of  survey  participants  providing  data  for  only  a 
portion  of  the  data  collection  period,  and  poststratification  to  more  accurate  population 
totals  from  the  Current  Population  Survey.  In  addition,  because  the  complex  sample 
design  is  different  from  simple  random  sampling,  estimates  of  variability  such  as 
standard  errors  and  confidence  intervals  requires  special  procedures  to  produce  national 
estimates  (Agency  for  Healthcare  Research  and  Quality,  2001).  AHRQ  recommends  not 
using  sample  sizes  less  than  100,  and  advise  that  survey  estimates  with  relative  standard 
errors  equal  to  or  greater  than  0.3  be  treated  as  unreliable.  Although  this  study  does  not 
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primarily  seek  to  produce  unbiased  national  estimates  with  the  data,  a  post  hoc  analysis 
will  be  conducted  and  compared  to  un-weighted  results. 

The  Medical  Provider  Component  (MPC)  supplements  and  validates  information  on 
medical  care  events  directly  by  telephone  and  mail-in  surveys  to  medical  providers 
identified  by  HC  respondents.  Medical  and  financial  characteristics  of  medical  events  are 
obtained  on  all  outpatient  department  and  emergency  room  visits,  hospital  stays,  home 
health  care  and  pharmacy  activities.  The  sampling  strategy  for  surveys  of  office-based 
physicians  providing  care  to  HC  respondents  was  slightly  different.  Zuvekas  (2001) 
reports  overall  sampling  rates  of  office-based  health  care  events  to  be  50  percent  in  the 
MPC.  The  office-based  MPC  included  all  physicians  of  HC  respondents  receiving 
Medicaid;  75  percent  of  households  receiving  care  under  an  HMO  or  other  managed 
care  plan;  and  25  percent  sample  of  remaining  households  (Agency  for  Healthcare 
Research  and  Quality,  1996). 

The  Prescription  Component  (PC)  describes  medications  prescribed  and  refilled 
during  each  round  to  include  prescription  date,  drug  name,  National  Drug  Code, 
quantity,  dosage  and  condition  for  which  the  medication  was  prescribed  (Moeller, 
Stagnitti,  &  Horan,  2001).  Although  medical  conditions  are  surveyed  as  part  of  the  HC, 
the  data  is  contained  in  a  separate  data  file  called  Medical  Conditions  (MC).  This 
research  uses  data  from  the  HC,  Medical  Conditions,  MPC  outpatient  department,  MPC 
office-based  provider  and  MPC  pharmacy  components.  For  each  calendar  year,  separate 
component  public  use  files  are  released  by  AHRQ.  For  Panel  1  Rounds  1-3  (1996)  there 
is  one  data  file  each  for  the  HC,  Medical  Conditions,  MPC  office-based  provider,  MPC 


71 


outpatient  department  provider  and  MPC  prescription  drugs.  In  subsequent  years,  each 
of  the  data  files  contains  two  overlapping  survey  panels.  To  estimate  a  single  year  of 
health  care  utilisation  and  expenditures,  there  is  no  distinction  between  the  overlapping 
panels,  but  to  construct  two  calendar  years  of  data,  such  as  for  this  study,  the  panels 
must  be  subset.  For  example,  to  construct  1996-1997  data,  Panel  1  rounds  1-3  for  1996 
must  be  combined  with  Panel  1  rounds  4-5  for  1997.  In  doing  so,  Panel  1  rounds  4-  5 
must  be  selected  while  discarding  Panel  2  rounds  1-3  reported  in  the  same  1997  file. 

Selecting  cases 

The  outcome  of  interest  for  this  study  is  the  number  of  visits.  All  reported  visits  in 
the  MEPS  were  subjected  to  a  three  criterion  test.  First,  the  visit  had  to  be  for  outpatient 
care.  This  criterion  was  met  by  drawing  visits  only  from  office-based  and  outpatient 
department  MEPS  files.  The  file  structure  of  office-based  and  outpatient  department 
files  are  identical  except  for  event  type.  To  simplify  case  selection,  office-based  and 
outpatient  department  files  were  merged  to  create  a  single  outpatient  visits  file  for  each 
survey  year.  Second,  the  visit  had  to  be  for  a  mental  health  purpose  or  condition.  Either 
it  had  to  be  coded  as  'psychotherapy’  on  one  of  two  variables,  or  there  had  to  be  a  Ninth 
Revision,  International  Classification  of  Disease  (ICD-9)  mental  health  diagnostic  code 
assigned  to  the  visit.  A  full  discussion  of  diagnostic  codes  is  provided  later  in  this 
section.  Third,  the  first  visit  meeting  the  above  criteria  had  to  occur  within  the  first  three 
rounds  of  Panel  1  for  1996-1997  and  Panel  3  for  1998-1999  to  coincide  with 
administration  of  satisfaction  with  care  questions.  If  the  satisfaction  survey  were  allowed 
to  precede  the  first  visit,  then  there  is  zero  possibility  that  level  of  satisfaction  could  be  a 
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product  of  experiences  with  outpatient  mental  health  care,  except  as  an  artifact  of  any 
previous  experiences. 

Once  selection  criteria  had  been  applied  to  the  four  outpatient  event  files,  and  mental 
health  visits  were  selected  based  on  their  first  occurrence  during  Rounds  1-3  of  Panel  1 
and  Panel  3  files,  person  ID5s  from  these  files  were  matched  to  additional  mental  health 
visits  that  occurred  in  Rounds  4-5  of  1997  and  1999  respectively.  No  new  ‘first  visits’ 
were  selected  in  the  second  year  of  data  collection  for  the  reasons  stated  in  the  previous 
paragraph.  Outpatient  visits  for  1996  Rounds  1-3  and  1997  Rounds  4-5  were  merged 
into  a  single  file  to  create  a  Panel  1  Rounds  1-5  data  file  containing  15,090  outpatient 
mental  health  visits.  Similarly  outpatient  visits  for  1998  rounds  1-3  and  1999  Rounds  4-5 
were  merged  to  create  a  single  data  file  of  7,854  outpatient  mental  health  visits  for  Panel 
3  Rounds  1-5.  As  expected  with  mental  health  utilization  data,  the  distribution  of  visits  is 
positively  skewed  with  46  percent  of  respondents  having  four  or  fewer  visits. 

Person  ID’s  from  the  selected  visits  were  matched  with  corresponding  ‘person  level5 
data  contained  in  the  HC  file.  Respondents  in  the  HC  without  a  matching  visit  were 
dropped  from  the  sample.  Recalling  there  is  one  HC  file  for  each  survey  year,  HC  files 
from  Panel  1  1996  and  1997  and  Panel  3  1998-1999  also  required  merging.  Although 
some  information  in  the  HC  files  did  not  change  from  year  to  year,  such  as  gender  and 
race,  there  were  other  variables  for  which  multiple  instances  had  to  be  retained,  such  as 
annual  expenditures,  reference  period  dates,  changes  in  insurance  coverage  and  round 
specific  variables  for  health  and  mental  health  status.  After  observations  with  non¬ 
matching  ID’s  were  dropped  and  multiple  occurrences  of  common  variables  had  been 
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uniquely  renamed,  the  four  HC  files  were  merged  to  form  a  single  HC  file  for  1996-1997 
and  a  single  HC  file  for  1998-1999. 

Dates  in  the  visits  files  were  examined  for  completeness.  Coded  as  ‘-9  Not 
ascertained’  or  ‘-8  Don’t  know’,  there  were  1129  missing  day  variables,  34  missing  month 
variables  and  one  missing  year.  This  presents  a  problem  for  using  time-to-event  methods 
which  requires  complete  date  values  for  conversion  to  julian  dates.  Precision  in 
reconstructing  the  exact  visit  date  is  not  necessary  since  the  analysis  calls  for  evaluating 
number  of  visits  until  failure-  not  intervals  between  visits  or  hazard  rates  in  real  time.  As 
a  result,  logical  edits  were  judged  appropriate  and  were  applied  to  missing  dates  using  the 
parameters  given  by  beginning  and  end  reference  period  dates  for  each  round.  If  day 
alone  was  missing  it  was  edited  as  the  15th  of  the  month.  If  more  than  one  day  value  was 
missing  in  the  same  month  for  a  single  respondent,  the  missing  values  were  replaced 
with  ‘15’  for  the  first  missing  day  value,  T6’  for  the  second  value,  T7’  for  the  third 
missing  value,  etc.  until  all  missing  values  for  that  month  had  been  replaced.  If  both  day 
and  month  values,  just  month  values,  or  just  year  values  were  missing,  the  missing  values 
were  edited  to  the  reference  period  begin  date  for  that  round. 

Because  the  study  seeks  to  capture  information  on  treatment  dropouts  and  because  it 
is  difficult  to  distinguish  dropouts  in  absolute  terms,  every  effort  is  needed  to  minimize 
the  appearance  of  failure  due  to  other  causes  such  as  truncated  data.  Truncation  occurs 
when  a  person  is  lost  to  follow-up  during  the  study  period  (W.Y.  Tsai,  personal 
communication,  September  16,  2002).  In  the  MEPS  truncation  can  occur  when  a 
respondent  is  out  of  scope,  or  ineligible  during  some  portion  of  the  study.  According  to 
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MEPS  documentation,  respondents  were  assessed  for  eligibility  at  each  interview  (2001). 
Ineligibility  could  be  temporary  due  to  such  factors  as  institutionalization  or  absence 
from  the  household,  or  ineligibility  can  be  permanent  due  to  death,  relocation  outside 
the  U.S.  or  withdrawal  for  some  other  reason.  If  a  respondent  has  one  or  more  mental 
health  visits,  is  out  of  scope  for  one  or  more  rounds,  but  then  is  in  scope  thereafter, 
what  looks  like  a  lag  time  between  visits  signifying  a  separate  episode  of  care  may  be  the 
result  of  truncation. 

Visual  inspection  identified  149  respondents  from  the  Panel  1  HC  file  and  64 
respondents  from  the  Panel  3  HC  file  as  having  some  kind  of  truncation  problem  where 
respondents  were  out-of-scope  for  at  least  part  of  one  round.  From  Panel  1  eight  had 
interruptions  in  eligibility  due  to  institutionalization,  14  joined  the  reporting  unit  after 
round  1  began,  four  were  deceased  and  one  moved.  From  Panel  3,  six  had  interruptions 
in  eligibility  due  to  institutionalization,  five  joined  the  reporting  unit  after  round  1  began, 
and  five  became  permanently  ineligible  because  of  relocation  or  death.  The  greatest 
number  of  truncated  cases  resulted  from  nonresponse.  There  were  91  respondents  from 
Panel  1  and  48  respondents  from  Panel  3  that  left  the  survey  after  Round  3  without  a 
disposition.  All  respondents  for  whom  truncation  is  noted  were  retained  in  the  data  with 
an  indicator  variable  created  to  flag  these  cases  for  later  analytic  purposes. 

A  similar  analytic  problem  with  longitudinal  data  is  right  and  left  censoring.  Right 
censoring  occurs  when  the  study  period  ends  before  a  respondent  has  experienced  the 
event  of  interest.  Left  censoring  occurs  when  the  event  of  interest  has  occurred  before 
the  study  period  begins.  According  to  Kleinbaum  (1996)  even  though  both  truncated 
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and  right  censored  observations  are  incomplete,  the  information  up  to  the  time  the  study 
ends  or  until  we  lose  track  of  the  person  can  still  be  used.  Left  censored  data  in  the 
context  of  this  study  is  more  worrisome  since  visits  may  have  occurred  before  the  study 
began  resulting  in  an  underestimation  of  number  of  visits.  There  are  two  options  for 
dealing  with  left  censored  data.  The  first  is  to  exclude  observations  that  are  likely  to  be 
left  censored  such  as  those  where  the  first  known  visit  occurred  early  in  the  study  period. 
L.  G.  Kessler  et  al.  (1980)  suggest  an  eight  week  interval  between  successive  visits  is  the 
maximum  lag  time  that  should  occur  without  signifying  a  new  episode.  Researchers 
investigating  depression  treatment  have  used  a  six  month  pretreatment  period  free  of  any 
indication  of  diagnosis  or  treatment  for  depressive  illness  to  identify  the  beginning  of  an 
episode(Croghan,  Obenchein,  &  Crown,  1998). 

The  eight  week  criterion  has  been  used  in  recent  research  (Bemdt  et  al.,  2000; 
Rosenthal,  2000).  However,  there  are  concerns  about  its  impact  on  sample  size  loss.  In 
the  NMES,  Hawkins  (1997)  estimated  that  eliminating  episodes  that  started  in  the  first 
eight  weeks  would  lead  to  a  42  percent  sample  loss  dataset.  This  is  particularly 
problematic  with  surveys  when  data  collection  begins  at  the  start  of  the  calendar  year,  a 
time  that  has  the  highest  incidence  of  people  starting  treatment  (Hawkins,  1997). 
Applying  the  eight-week  interval  to  this  study  would  mean  that  all  respondents  who  had 
a  mental  health  visit  within  eight  weeks  of  the  beginning  of  data  collection  would  be 
excluded.  Thirty-eight  percent  of  the  Panel  1  sample  and  34  percent  of  the  Panel  3 
sample  would  be  lost.  Although  leaving  left  censored  observations  in  the  sample 
systematically  underestimates  number  of  visits,  it  is  a  trade-off.  Round  1  observations  are 
most  likely  to  be  censored,  but  these  are  also  the  observations  that  temporally  precede 
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administration  of  the  satisfaction  survey  during  Round  2.  Retaining  these  is  highly 
desirable  even  at  the  expense  of  underestimating  number  of  visits.  A  flag  variable  is 
created  for  left  censored  cases  to  use  for  analytic  purposes. 

Selecting  variables 

Satisfaction  with  care 

This  section  describes  the  handling  of  missing  data  on  satisfaction  with  care  variables, 
testing  of  the  satisfaction  measure  and  construction  of  importance  weights  to  minimize 
the  influence  of  satisfaction  responses  derived  from  health  not  mental  health 
experiences.  A  table  of  descriptive  statistics  for  satisfaction  variables  is  available  in  Table 
17.  As  discussed  in  the  literature  review  satisfaction  with  care  has  been  conceptualized  as 
either  a  unidimensional  and  a  multidimensional  construct  with  the  strongest  empirical 
support  pointing  to  its  use  as  a  multidimensional  construct.  For  this  study,  a 
multidimensional  measure  of  satisfaction  with  care  is  constructed  using  survey  questions 
from  the  Access  to  Care  supplement  of  the  HC.  Although  no  respondents  were 
identified  as  ineligible  for  the  supplement,  there  were  still  between  139  and  211  missing 
observations  resulting  from  questions  not  asked  due  to  skip  patterns  in  the  interview. 
Scored  as  ‘4  Inapplicable",  the  first  skip  pattern  resulted  in  139  missing  on  five 
satisfaction  questions.  A  second  skip  pattern  resulted  in  211  missing  on  one  other 
question  for  ‘appointment  difficulty".  By  eliminating  the  139  as  not  belonging  in  either 
numerator  or  denominator,  it  reduced  the  missing  on  five  variables  to  zero,  but  72 
missing  remain  on  ‘appointment  difficulty". 
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There  were  three  ways  of  handling  this  problem.  One  method  would  be  to  eliminate 
the  full  211  which  would  result  in  no  missing  values  on  any  of  the  six  satisfaction 
variables  and  would  provide  a  remaining  sample  size  of  995,  but  at  the  cost  of  losing  72 
non-missing  observations  on  the  four  variables  that  were  only  missing  139  values.  The 
second  method  would  be  to  eliminate  the  139  observations,  and  drop  the  variable 
‘appointment  difficulty’  from  the  survey.  This  would  result  in  five  remaining  satisfaction 
variables  and  provide  a  sample  size  of  1067,  but  it  creates  another  problem.  If 
‘appointment  difficulty’  is  eliminated,  the  second  variable  ‘difficulty  contacting  provider 
by  phone’,  is  no  longer  an  adequate  conceptual  representation  of  ‘access’  and  must  also 
be  eliminated  leaving  four  satisfaction  variables.  The  third  method  is  to  drop  the  139 
observations,  but  attempt  to  retain  ‘appointment  difficulty’  and  ‘telephone  difficulty’  in 
spite  of  missing  values.  The  third  method  was  chosen. 

The  satisfaction  measure  consists  six  questions  hypothesized  to  represent  four 
underlying  dimensions  of  satisfaction:  access  (two  items),  patient-provider  interaction 
(two  items),  satisfaction  with  staff  (one  item)  and  satisfaction  with  quality  of  care  (one 
item).  The  goal  is  to  devise  a  measure  from  these  six  variables  which  accounts  for  the 
most  variance,  but  which  produce  the  least  redundancy,  or  shared  variance.  To  explore 
the  relationship  between  the  variables,  a  correlation  matrix  was  produced  which 
indicated  ‘satisfaction  with  provider/staff  and  ‘satisfaction  with  quality  of  care’  are 
moderately  correlated  (r  =  .58)  and  share  .22  of  the  variance  (average  inter-item 
covariance).  Satisfaction  with  quality  of  care  and  confidence  in  provider  were  correlated 
at  r=.55.  Confidence  in  provider  and  provider  listens  were  correlated  at  r=.50. 
Appointment  difficulty  and  phone  difficulty  was  .42.  Confirmatory  factor  analysis  was 
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used  to  evaluate  the  feasibility  of  a  four  factor  model.  Principal  components  analysis 
with  varimax  rotation  indicated  that  the  six  variables  measure  unique  aspects  of 
satisfaction  and  should  not  be  combined  into  subscales  as  planned. 

The  second  stage  in  constructing  the  satisfaction  measure  was  to  develop  importance 
weights  that  maximize  the  influence  of  respondents  with  greater  mental  health  utilization 
and  minimize  the  influence  of  respondents  with  greater  non-mental  health  utilization. 
Respondents  are  likely  to  base  their  satisfaction  evaluations  on  that  with  which  they  have 
had  more  experience.  Therefore,  respondents  with  a  higher  ratio  of  mental  health  to 
non-mental  health  utilization  are  likely  to  have  based  satisfaction  evaluations  on  their 
experience  with  mental  health  care.  Respondents  with  a  low  ratio  of  mental  health  to 
health  utilization  are  likely  to  have  based  satisfaction  evaluations  on  non-mental  health 
care.  There  were  several  options  developing  importance  weights.  The  first  method  was 
to  use  the  ratio  of  mental  health  to  non-mental  health  expenditures.  The  advantage  is 
that  more  complex  encounters  in  which  there  is  more  contact  with  provider  and 
provider  staff  will  have  greater  costs  and  will  count  more.  However,  there  are  two 
disadvantages. 

First,  the  range  of  potential  costs  associated  with  outpatient  mental  health  visits  is  less 
than  the  range  of  potential  costs  associated  with  non-mental  health  care  visits;  there  are 
only  a  limited  number  of  diagnostic  tests  and  treatments  routinely  associated  with 
outpatient  mental  health  care.  The  other  disadvantage  is  that,  although  expenditures 
reflect  the  cost  of  care,  with  insurance  most  individuals  do  not  bear  the  full  cost.  The 
second  method  was  to  use  the  ratio  of  out-of-pocket  mental  health  to  out-of-pocket 
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non-mental  health  expenditures.  Out-of  pocket  payments  have  the  advantage  of  better 
reflecting  patient  opportunity  costs.  The  disadvantage  is  out-of  pocket  payments  are 
conditional  on  insurance  plan,  and  many  insurance  plans  have  better  coverage  in  the 
form  of  lower  out-of-pocket  costs  for  non-mental  health  care  over  mental  health  care. 
The  third  method  was  to  use  the  ratio  of  frequency  counts  of  mental  health  visits  to 
non-mental  health  visits  where  only  the  quantity  of  visits  matters.  The  advantage  is  every 
visit  of  each  type  counts  equally  regardless  of  how  costs  are  distributed.  They 
disadvantage  is  that  visit  counts  do  not  account  for  patient  sacrifice  other  than  time 
costs. 

Because  the  choice  is  not  straightforward,  all  three  types  of  importance  weight  were 
evaluated.  Variables  were  created,  each  a  ratio  of  mental  health  to  all  health  care 
expenses,  out-of-pocket  payments  or  visit  counts.  A  clear  advantage  to  visit  weights  was 
revealed  from  the  fact  that,  unlike  the  ratio  of  out-of-pocket  costs,  the  ratio  of  mental 
health  to  all  health  care  visits  can  never  be  zero  because  there  will  always  be  at  least  one 
visit.  Although  one  might  expect  the  three  types  of  weights  to  be  highly  correlated  since 
they,  in  theory,  measure  the  same  thing,  the  correlation  coefficient  suggests  a  more 
moderate  relationship.  The  strongest  correlation  was  between  visit  weights  and 
expenditure  weights  (^.68).  Expenditure  weights  and  out-of-pocket  payment  weights 
were  correlated  at.59.  The  correlation  between  visit  weights  and  out-of-pocket  payment 
weights  was  the  lowest  (r=.40).  Next  the  distribution  of  the  three  weights  of  the 
population  were  examined.  Both  expenditure  weights  and  out-of-pocket  payment 
weights  were  positively  skewed  with  out-of-pocket  payment  weights  showing  the  most 
extreme  distribution.  Many  respondents  had  close  to  zero  out-of  pocket  payments  for 
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mental  health  care  even  among  the  uninsured  group.  The  distribution  of  expenditure 
weights  was  positively  skewed  but  with  more  of  a  curvilinear  shape.  Visit  weights  also 
displayed  a  curvilinear  shape,  but  without  the  skewness  seen  with  the  other  two  weights. 
Because  of  the  skewed  distributions  for  expenditure  and  out-of-pocket  weights,  the 
means  were  also  different  .  The  visit  weights  were  favored  over  out-of-pocket  and 
expenditure  weights. 

Sociodemographic  Covariates 

This  section  discusses  selection  of  sociodemographic  variables  to  support  the  three 
study  hypotheses  to  include  selection  and  construction  of  insurance  variables.  A  table  of 
descriptive  statistics  for  sociodemographic  variables  is  available  in  Table  18.  Several 
sociodemographic  variables  served  as  covariates:  age,  race,  gender,  education  and 
insurance.  Each  was  chosen  for  its  empirical  importance  in  influencing  treatment 
duration  and/or  patient  satisfaction  and  thought  to  confound  the  hypothesized 
relationship  between  satisfaction  with  care  and  number  of  visits.  The  covariates  are  not 
exhaustive  and  there  is  always  the  risk  of  outside  factors  influencing  the  data  which  are 
not  measured.  Coding  on  the  selected  covariates  in  MEPS  is  straightforward.  Age  is  a 
continuous  variable  recoded  categorically  for  ‘0-4’,  ‘5-17’,  ‘18-24’,  ‘25-44’,  ‘45-64’,  ‘65-90’. 
Five  missing  values  on  age  were  replaced  with  the  sample  mean.  Children  were  retained 
in  the  sample,  but  ages  0-4  are  unlikely  to  have  an  outpatient  mental  health  visit  and 
more  likely  to  have  parents/caretaker  responses  to  satisfaction  questions.  Nonetheless 
there  is  no  empirical  evidence  to  suggest  children  should  be  excluded.  Gender  is  coded 
as  ‘M’  or  ‘F’.  There  are  two  variables  for  race  in  the  MEPS:  Race  includes  ‘American 
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Indian’,  ‘Aleut/ Eskimo’,  ‘Asian  or  Pacific  Islander’,  ‘Black’,  ‘White’  and  ‘Other’.  The 
sample  size  for  American  Indian,  Aleut/Eskimo  and  Asian  or  Pacific  Islander  was  less 
than  5  percent  of  the  total  sample  and  would  result  in  small  cell  size.  As  a  result  the 
second  race  variable:  Race/Ethnicity  was  selected:  ‘Person  is  Hispanic’,  ‘Person  is 
Black/Not  Hispanic’,  ‘Other’.  This  will  capture  the  two  largest  non-white  ethnic  groups: 
Black  and  Hispanic.  The  education  variable  has  eight  levels  in  the  MEPS  which  were 
collapsed  to  four  levels  for  this  study:  ‘No  degree’,  ‘High  school  diploma/GED’, 
‘Bachelor’s /Other  degree’,  ‘Master’s/Doctorate  degree’,  ‘Under  16-  Inapplicable’. 
Missing  data  was  infrequent  on  demographic  variables,  but  when  a  missing  value 
occurred,  the  sample  mean  was  used  and  flag  variables  created  for  the  category  of 
missing  (‘don’t  know’  and  inapplicable’)  for  analytic  testing  if  needed 

There  are  over  300  variables  in  the  MEPS  pertaining  to  insurance  coverage.  For  this 
study  it  is  be  important  to  know  1)  whether  the  respondent  had  health  insurance  and  2) 
whether  changes  in  coverage  occurred  over  the  24  month  study  period.  Respondents 
who  have  no  insurance,  or  who  experience  disruptions  in  coverage,  would  likely  make 
different  decisions  about  continuation  in  mental  health  care.  Monheit  et  al.  (2001) 
reports  92.0  percent  of  MEPS  respondents  had  full-year  private  health  insurance 
coverage  for  1996.  The  rate  was  slightly  lower  for  African  Americans  (87.0)  and 
Hispanics  (88.9).  In  this  sample  86  percent  had  full-year  private  coverage;  78  percent  of 
African  Americans  and  85  percent  of  Hispanics.  Monheit  et  al.  report  75  percent  of 
individuals  with  public  insurance  had  full-year  coverage  in  1996.  In  this  study  the  rate 
was  also  75  percent.  They  found  poor  people  have  greater  stability  in  their  public 
coverage  than  any  other  income  group  except  high  income.  One  fifth  of  those  with 
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public  insurance  became  uninsured  at  some  point  in  the  year  and  only  70  percent  of 
people  uninsured  at  the  beginning  of  the  year  acquired  insurance  later  in  the  year. 

Four  insurance  variables  are  used  for  this  study:  COVRG123  and  COVRG45  are 
categorical  variables  with  three  levels  for  T  private  insurance  coverage’,  ‘2  public 
insurance  coverage’  and  c3  uninsured’.  Two  new  variables  INSCHANG  and  RINSUR 
were  constructed  based  on  the  extent  of  public  or  private  health  insurance  coverage 
extending  over  the  24  month  data  collection  period.  RINSUR  is  the  ratio  of  insured  to 
uninsured  months  and  INSCHNG  is  a  categorical  variable  derived  from  RINSUR:  T’  is 
18-24  months  coverage;  ‘2’  is  12  -  17  months  coverage;  ‘3’  is  6  -  11  months  coverage  ; 
and  c4’  is  0-6  months  coverage.  There  were  96  and  48  missing  observations  for  insurance 
coverage.  A  cross-check  found  all  cases  flagged  for  truncation  which  indicated  those 
respondents  were  out  of  scope  for  at  least  part  of  one  round. 

Impairment  Covariates 

Severity  of  illness  is  expected  to  influence  decisions  about  the  duration  of  mental 
health  care  through  respondent  perceptions  of  severity  of  illness  and  provider 
assessment  and  recommendations  about  the  type  and  intensity  of  care  needed.  This 
section  provides  a  discussion  of  impairment  variable  selection,  to  include  methods, 
strengths  and  limitations  of  selected  variables.  A  table  of  descriptive  statistics  for 
impairment  variables  is  available  in  Table  21.  MEPS  variables  that  address  the  severity  of 
illness  do  so  in  only  two  ways.  First,  through  a  diagnosis  and  second,  through  self 
reported  perceptions  of  illness  such  as  health,  mental  health  status  and  activity  and  social 
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limitations.  Diagnosis  in  the  MBPS  was  attained  by  recording  respondent  descriptions  of 
medical  conditions  which  were  then  converted  by  professional  coders  to  ICD-9CM 
diagnostic  codes  (Agency  for  Healthcare  Research  and  Quality,  1996).  The  precision  of 
diagnosis  is  subject  to  the  ability  of  the  respondents  to  report  conditions  that  could  be 
coded  accurately.  For  confidentiality,  all  MEPS  diagnoses  were  collapsed  to  three  digits 
of  five  digits  in  public  use  files.  Mental  health  diagnoses  selected  for  this  study  are  shown 
in  Table  20  and  exclude  substance  abuse  disorders.  Individuals  with  substance  abuse 
disorders  may  face  a  very  different  demand  curve  for  treatment  than  individuals  with 
other  mental  disorders;  treatment  is  often  involuntary  and  the  decision  to  comply  based 
on  more  complex  factors  outside  the  scope  of  this  study.  Except  for  depressive 
disorders,  the  numbers  of  respondents  with  bipolar  disorders,  psychotic  disorders 
among  others,  is  small.  For  this  study  I  am  most  interested  in  whether  a  respondent  is 
being  seen  has  a  mental  health  diagnosis,  a  medical  diagnosis  only  or  no  diagnosis,  but  I 
acknowledge  the  severity  of  illness  and  rational  behavior  that  can  be  expected  from  for 
example  an  individual  having  a  psychotic  disorder  is  quite  different  from  an  individual 
having  an  adjustment  disorder.  Not  controlling  for  each  diagnosis  is  a  study  weakness. 

There  are  variables  for  diagnosis  at  both  the  condition  (HC)  and  the  visit  level. 
Because  the  unit  of  analysis  for  this  study  are  ‘visits’,  visit-level  diagnoses  were  used.  This 
allowed  selection  of  cases  where  there  was  a  mental  health  visit  even  when  no  diagnosis 
was  made,  and  to  exclude  cases  where  there  was  a  condition  but  no  related  visits.  The 
disadvantage  to  using  visit  level  diagnoses  is  that  an  individual  may  have  many  visits  and 
as  many  as  four  diagnoses  for  each.  This  ambiguity  is  different  from  evidence  of 
comorbidity,  where  two  or  more  mental  health  conditions  are  present  and  under 
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treatment  simultaneously.  Comorbidity  would  appear  as  two  or  more  ICD-9  diagnoses 
together  in  the  same  visit  or  visits. 

To  address  the  problem  of  ambiguous  diagnoses,  hand  selection  of  the  first  mental 
health  diagnosis  coinciding  with  the  earliest  visit  was  made.  If  comorbidity  was  evident 
then  the  diagnosis  in  the  primary,  or  first  position,  was  selected.  If  no  diagnosis  appeared 
then  the  person  was  coded  ‘no  diagnosis’.  If  only  medical  diagnoses  appeared  across 
visits,  the  respondent  was  coded  as  having  a  ‘medical  diagnosis  only’.  From  the  Panel  1 
files  there  were  740  respondents  with  a  mental  health  diagnosis;  216  respondents  with 
only  a  medical  diagnosis;  111  respondents  with  neither  a  mental  health  nor  medical 
diagnosis;  and  four  for  whom  diagnosis  was  not  known  (eventually  replaced  with  the 
modal  value  of  ‘311’).  From  the  Panel  2  files  there  were  420  respondents  with  a  mental 
health  diagnosis;  99  respondents  with  only  a  medical  diagnosis;  and  38  with  neither  a 
mental  health  nor  a  medical  diagnosis. 

To  validate  the  hand  selection  of  diagnosis,  ICD9  codes  from  the  MC  file  were 
matched  with  event-level  codes  described  above  to  examine  the  data  for  discrepancies. 
Recall  that  MEPS  respondents  identified  existing  medical  conditions,  independent  of 
conditions  specifically  associated  with  health  care  events  or  visits.  From  the  1,067 
respondents  in  Panel  1  mental  health  visit  files,  there  were  89  (.08)  discrepancies  between 
visit  and  condition  diagnosis:  52  discrepancies  where  there  was  a  mental  health 
condition,  but  only  general  medical  visit-level  diagnosis;  nine  discrepancies  where  there 
were  two  different  mental  health  diagnoses;  and  one  mental  health  condition  where  the 
visit-level  diagnosis  was  coded  as  ‘don’t  know’.  The  diagnosis  coded  as  ‘don’t  know’  was 
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replaced  with  the  condition-level  diagnosis.  In  the  match  between  medical  conditions 
and  event-level  diagnosis  for  the  Panel  3  files,  of  the  557  respondents  there  were  28 
(0.05)  discrepancies:  17  discrepancies  where  there  was  an  mental  health  condition,  but 
only  general  medical  visit-level  diagnosis;  five  where  there  were  two  different  mental 
health  diagnoses;  and  six  where  there  was  a  mental  health  condition  but  the  visit-level 
diagnosis  was  coded  ‘no  diagnosis’.  Additional  discrepancies  in  diagnoses  were  found 
when  depression  cases  (below)  were  evaluated  using  MEPS  data  center  non-public  use 
files  suggesting  underestimation  of  sample  wide  depression  cases  in  this  study. 

The  third  test  hypothesis  for  this  study  requires  a  subgroup  of  patients  diagnosed 
with  depression.  However,  because  of  the  three  digit  collapsing,  additional  measures  are 
required  to  parcel  out  depression  from  diagnostic  categories  ‘296’  and  ‘300’.  In  their 
study  on  national  trends  in  outpatient  treatment  of  depression  Croghan  et  al  (1998)  and 
Olfson  et  al.(2002)  used  296.2  major  depressive  disorder,  single  episode;  296.3  major 
depressive  disorder;  300.4  dysthymic  disorder,  recurrent;  and  311  depressive  disorder, 
not  otherwise  specified  to  define  depression.  Using  this  same  categorization  required  the 
fourth  digit  of  diagnostic  codes  available  by  accessing  non-public  use  files  direcdy  from 
the  AHRQ  data  center. 

Respondent  perceptions  of  severity  of  illness  was  constructed  by  using  several  MEPS 
variables:  perceived  health  status,  perceived  mental  health  status  and  work,  activity, 
cognitive  and  social  limitations.  Perceived  health  status  is  included  based  on  previous 
research  findings  that  chronic  illness  is  often  a  factor  in  mental  health  utilization.  For 
example,  in  the  Panel  1  data  21  percent  of  individuals  with  a  mental  health  visit  had  a 
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medical,  but  no  mental  health  diagnosis.  The  current  data  suggested  the  correlation 
between  ‘perceived  health  status5  and  ‘perceived  mental  health  status5  variables  might  be 
too  high  to  use  separately  (r=.51).  This  occurred  only  in  the  depression  subgroup 
analysis  and  was  addressed  by  omitting  mental  health  status  since  most  in  the  depression 
subgroup  were  known  to  have  poor  mental  health  status.  Perceived  health  status  and 
mental  health  status  are  each  categorical  variables  measured  on  five  levels:  ‘excellent5, 
‘very  good5,  "good5,  ‘fair5  and  ‘poor5.  A  problem  is  that  these  are  multi-round  variables 
that  must  be  reduced  to  avoid  the  autocorrelation  problem  characteristic  of  time  series 
data.  There  are  three  options  for  reducing  these  to  two  single  variables.  Some  MEPS 
researchers  collapsed  health  status  categories  to  three  broader  categories:  1)  excellent, 
very  good  or  good  all  year,  2)  fair  or  poor  all  year  and  3)  ever  in  fair  or  poor  health 
during  the  year  (Monheit  et  al.,  2001).  A  second  option  is  to  use  the  average  response 
over  five  rounds.  The  difficulty  with  this  approach  is  that  for  most  respondents,  mental 
health  visits  will  not  extend  over  five  rounds.  Assuming  respondents  report  better  health 
and  mental  health  status  both  before  and  after  receiving  care,  the  average  response  will 
result  in  less  variation  than  a  point  in  time  value.  A  third  option  is  to  select  the  instance 
that  coincides  with  the  round  of  the  first  mental  health  visit.  This  captures  the  degree  of 
impairment  at  the  point  of  care  which  is  more  desirable  than  the  other  two  options. 
Missing  values  are  resolved  by  carrying  forward  the  last  non-missing  observation  when 
possible.  In  five  cases  this  was  not  possible  and  these  remain  missing  (Table  21). 

Other  impairment  variables  available  in  the  MEPS  are  activity  limitations,  social 
limitations  and  cognitive  limitations.  Although  activity  and  social  limitations  are  coded 
for  all  respondents,  cognitive  limitations  is  missing  almost  200  observations  as 
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‘inapplicable’.  A  closer  examination  indicates  these  were  children  for  whom  the  question 
was  not  appropriate.  No  substitute  question  was  available  so  this  variable  was  dropped 
altogether  from  the  study.  ‘Activity  limitations’  and  ‘social  limitation’s  are  also  multi¬ 
round  variables  assessed  during  Rounds  1,  3  and  5.  As  before,  the  round  coinciding  with 
the  first  visit  was  used.  There  were  many  more  missing  values  because  of  the  skip  pattern 
discussed  earlier  making  the  variables  inapplicable  for  some  respondents.  A  correlation 
matrix  reveals  a  moderate  correlation  between  activity  and  social  limitations  at  .57. 
Multicollinearity  was  be  addressed  by  omitting  social  limitations.  This  occurred  only  in 
the  depression  subgroup  analysis. 

Constructing  compliance  measure 

Although  psychotherapy,  is  the  focus  of  this  study,  it  is  not  the  only  form  of 
outpatient  treatment,  and  in  fact  is  not  even  the  most  common.  Rates  of  pharmacologic 
treatments  for  many  conditions,  such  as  depression  and  anxiety  is  rapidly  increasing. 
(Olfson  et  al.,  2002)  estimates  that  almost  80  percent  of  patients  treated  for  depression  in 
1997  received  a  prescription  for  an  antidepressant.  Mental  health  treatment  may  be 
carried  out  using  some  combination  of  psychotherapy  and  medication,  depending  on  the 
condition,  clinical  setting  and  provider.  If  a  respondent  has  ceased  to  have  mental  health 
visits,  but  is  still  taking  psychotropic  medication,  then  they  have  not  really  dropped  out 
of  treatment.  Therefore,  the  distinction  is  important.  This  section  provides  a  description 
of  the  method  used  to  construct  a  measure  of  compliance  for  the  depression  treatment 
compliance  subgroup  that  uses  a  combination  of  visits  and  medications. 
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To  construct  this  information,  several  variables  from  the  outpatient  event  and 
prescription  drug  file  were  needed.  Medication  name  and  national  drug  code  are  used  to 
derive  the  name  of  the  medication.  Because  some  drugs  have  both  psychiatric  and  non¬ 
psychiatric  applications,  ICD-9  diagnosis  from  the  prescription  drug  file  were  used  in 
concert  with  drug  name  to  ensure  the  drug  is  being  used  in  the  treatment  of  a  mental 
health  problem.  Using  the  previous  list  of  ICD-9  codes  to  establish  a  mental  health 
diagnosis  there  were  13,977  drug  events  for  925  respondents.  Of  those  respondents  with 
any  mental  health  diagnosis,  there  were  11,445  drug  events  for  842  respondents  that 
involved  psychotropic  medications.  To  isolate  respondents  being  treated  for  depression, 
the  criteria  was  further  narrowed  to  those  respondents  on  a  psychotropic  drug  with 
diagnoses  of  ‘296.2/3’  ‘300.4’  ‘31 T  yielding  9,979  psychotropic  drug  events  for  791 
respondents.  Finally,  respondents  being  treated  for  depression  and  taking  at  least  one 
antidepressant  medication  were  selected  yielding  5,115  drug  events  for  650  respondents. 
Once  it  was  determined  that  a  relevant  psychotropic  drug  existed,  a  dichotomous 
variable  was  created.  A  second  data  file  was  created  for  the  650  respondents  with  at  least 
one  antidepressant  drug  event  containing  the  drug  type  and  maximum  number  of 
prescriptions  for  each  drug  type. 

The  next  question  was  how  many  prescriptions  for  an  antidepressant  were  needed  to 
establish  that  minimal  treatment  occurred  in  lieu  of  treatment  visits.  Four  prescriptions 
for  any  antidepressant  filled  within  six  months  of  treatment  initiation  has  been 
previously  used  as  a  proxy  for  minimally  effective  treatment  published  in  AHRQ 
guidelines  (Melfi  et  al.,  1998;  Tai-Seale  et  al.,  2000).  In  this  research  the  four  prescription 
minimum  will  serve  as  a  proxy  for  those  who  minimally  met  medication  treatment 
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recommendations.  Using  this  criteria  26  percent  of  the  650  respondents  diagnosed  with 
a  depressive  disorder  who  were  taking  an  antidepressant  medication  had  at  least  four 
prescriptions  making  them  compliant  irrespective  of  number  of  visits. 

To  prepare  a  model  to  test  the  hypothesis  of  a  relationship  between  satisfaction  with 
care  and  the  probability  of  depression  treatment  compliance  a  subgroup  of  respondents 
with  a  diagnosis  of  ‘296.2’,  ‘296. 3’  major  depressive  disorders;  ‘300.4  neurotic  depression’ 
and  ‘311  depressive  disorders’  was  selected  (N=967)\  Of  the  967  respondents  with  a 
diagnosis  of  depression,  650  received  at  least  one  prescription  for  antidepressant  drug 
and  168  received  at  least  four  prescriptions.1 2  Based  on  the  four  prescription  or  more 
than  five  visit  criteria  to  establish  depression  treatment  compliance,  a  total  of  619 
respondents  were  deemed  completely  compliant.  This  left  348  respondents  with  a 
diagnosis  of  depression  who  were  noncompliant. 


1  Depression  cases  accessed  from  condition  files  at  the  MEPS  data  center  yielded  a  slightly  large 
number  of  respondents  because  of  differences  in  case  selection  criteria. 

2  The  remaining  3 17  of  967  were  either  not  on  any  psychotropic  medication  or  were  on  a  non¬ 
antidepressant  psychotropic  medication. 


90 


CHAPTER  5  Results 

This  chapter  is  comprised  of  two  sections.  The  bivariate  analysis  section  describes 
basic  characteristics  of  the  data  with  two  goals  in  mind.  The  first  goal  is  to  describe 
utilization  rates  across  the  sample  population  in  order  to  identify  patterns  in  the  average 
number  of  visits  and  to  describe  changes  in  the  proportions  of  users  at  each  level  of  use. 
Two  sets  of  tables  are  provided  in  this  section  to  show  the  average  number  of  visits 
across  the  sample  population  and  sample  population  changes  occurring  over  the  first  15 
visits.  The  second  goal  of  the  bivariate  section  is  to  describe  chi  square  tests  of 
association  and  univariate  regression  models  that  establish  fundamental  relationships 
between  variables.  The  purpose  is  to  understand  how  and  if  predictor  and  control 
variables  are  independendy  related  to  the  study  outcomes,  and  how  and  if  predictor  and 
control  variables  are  related  to  each  other.  Two  tables  of  bivariate  associations  are 
provided  for  associations  in  the  first  four  visits  and  associations  in  the  depression 
treatment  compliance. 

The  second  section  of  the  results  chapter  describes  the  analytic  models  to  address 
the  three  research  hypothesis.  Each  section  describes  main  effects  and  interaction 
models  and  is  organized  for  each  of  the  six  satisfaction  with  care  variables.  The  first 
models  evaluate  the  relationship  between  satisfaction  with  care  and  the  probability  of  a 
subsequent  visit.  One  table  of  main  effects  for  the  probability  of  more  than  one  visit  is 
provided  with  summary  tables  of  significant  interactions  for  each  satisfaction  variable. 
The  second  models  evaluate  the  relationship  between  satisfaction  with  care  and  the 
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probability  of  depression  treatment  guideline  compliance.  Tables  are  provided  for  full 
and  reduced  main  effects  models  and  significant  interaction  terms.  The  third  models  use 
survival  analysis  to  evaluate  differences  in  time  to  failure  rates  between  high  and  low 
satisfaction  with  care.  Figures  of  Kaplan-Meir  curves,  adjusted  survival  curves  and  a 
summary  table  of  stratified  Weibull  regression  models  is  provided  in  Appendix  C. 

Bivariate  statistics 

This  section  explores  bivariate  associations  between  satisfaction  with  care,  covariates 
and  number  of  visits.  First,  high  and  low  utilizers  of  outpatient  mental  health  care  were 
identified  by  examining  the  mean  number  of  visits  across  the  sample  and  within  the 
depression  subgroup.  Second,  the  data  was  examined  for  differences  in  the  population 
between  those  who  dropped  out  early,  those  who  remained  in  care  longer,  and  those 
who  were  part  of  the  depression  subgroup.  Third,  contingency  tables  were  used  to 
describe  bivariate  associations  between  each  of  the  satisfaction  and  control  variables  and 
the  probability  of  more  than  one,  two  and  three  visits,  and  probability  of  compliance  for 
the  depression  subgroup.  Chi-square  statistics  confirm  the  statistical  importance  of  each 
variable  providing  support  for  variable  selection  in  multivariate  models.  Fourth, 
covariates  were  used  in  univariate  regression  models  to  test  for  potential  confounding 
effects  that  can  occur  when  a  covariate  is  related  to  both  a  predictor  variable  and  an 
outcome  variable. 

Mean  differences  in  visits 

I  examined  the  data  to  determine  the  average  number  of  visits.  Table  22  (Appendix 
A)  indicates  individuals  in  excellent  mental  health,  with  no  diagnosis,  age  groups  18-24 
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and  65-90,  and  individuals  who  were  not  at  all  satisfied  with  quality  of  care  had  the 
fewest  number  of  visits.  Individuals  in  poor  mental  and  physical  health,  ages  0-17  and 
25-44,  college  education,  publicly  insured,  and  individuals  with  social  and  activity 
limitations  had  the  highest  number  of  visits.  Analysis  of  variance  was  used  to  test  for 
differences  in  mean  number  of  visits  and  revealed  significance  differences  for  health, 
mental  health,  activity  limitations,  social  limitations,  and  psychotropic  medication. 

Within  the  depression  subgroup  there  were  also  significant  differences  in  number  of 
visits  (Table  23  Appendix  A).  Individuals  in  the  age  group  65-90,  with  less  than  a  high 
school  education,  in  excellent  mental  health  and  those  not  at  all  satisfied  with  quality  of 
care  had  the  fewest  number  of  visits.  Individuals  with  an  advanced  degree,  under  16 
years  old,  publicly  insured,  reporting  excellent  health  and  poor  mental  health  and 
individuals  not  at  satisfied  with  provider/ staff  had  the  highest  average  number  of  visits. 
Analysis  of  variance  revealed  statistically  significant  differences  for  education;  health 
status;  mental  health;  and  social  limitations3. 

Population  differences 

While  it  is  helpful  to  know  the  average  number  of  visits  across  the  population,  this 
does  not  reveal  when,  how  and  if  the  population  changes  as  respondents  dropped  out  of 
care.  The  three  analytic  models  for  this  research  examine  the  relationship  between 
satisfaction  with  care  and  the  probability  of  a  subsequent  visit  capturing  three  different 
visit  intervals  and  potentially  three  different  populations  of  users.  While  the  first  four 
visits  are  the  focus  of  the  first  model,  the  second  model  examines  differences  in  more 


3  ANOVA  performed  before  dropping  non-depression  ‘296.X’  and  ‘31  l.X5  observations 
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than  five  and  less  than  five  visits  and  the  third  examines  the  hazard  rate  across  all  visits. 
This  could  generate  questions  about  whether  differences  in  the  effect  of  satisfaction  on 
the  probability  of  a  subsequent  visit  is  the  result  of  real  differences  or  simply  changes 
that  have  occurred  in  the  population  as  individuals  dropped  out  of  care.  The  sample  was 
examined  for  changes  in  the  population  across  intervals  of  visits  to  see  if  there  were 
significant  differences  in  the  population  that  could  influence  the  findings.  In  support  of 
the  first  model,  changes  in  proportions  were  examined  between  respondents  having 
more  than  one,  two  and  three  visits  (Table  1  below). 


Table  1  Sample  differences  between  more  than  one,  two  and  three  visits 


MT1  visit 


MT2  visit 


Mental  health 
Medical  diagnosis  only 
No  diagnosis 


Psychotropic  drug 

No 

Yes 


Health  status 

Excellent/very/good 

Fair/poor 


Mental  health 

Excellent/very/good 

Fair/poor 


Activity  limitation 

No 

Yes 


Social  limitations 

No 

Yes 


Appt  difficulty 

No 

Yes 


Phone  difficulty 

No 

Yes 


Provider  listens 

Yes 

No 


Provider  confidence 

Yes 

No 


Satisfied  with  provider/staff 

Satisfied 

Not  satisfied 


Satisfied  with  quality  of  care 

Satisfied 

Not  satisfied 


MT3  visit 


73.46 

72.5 

26.54 

27.4: 
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For  the  second  model,  the  depression  treatment  guideline  compliant  group,  I  wanted 
to  know  whether  differences  in  the  guideline  compliant  group  were  due  to  having  more 
than  five  visits  (from  which  the  compliance  measure  is  constructed)  or  whether 
differences  in  the  population  were  the  result  of  selection  into  the  depression  subgroup. 
Changes  in  proportions  were  examined  between  respondents  who  were  treatment 
guideline  compliant,  all  respondents  who  had  more  than  five  visits  and  only  respondents 
with  a  depression  diagnosis  who  had  more  than  five  visits  (Table  2  below). 


Table  2  Sample  differences  at  five  visits,  for  depressed  subgroup  at  five  visits  and  for 
respondents  who  were  compliant  with  depression  treatment  guidelines 


Sample  at 
MT5  visits 

Depression 
subgroup 
MT  5  visits 

Guideline 
at  compliant 
group 

Sample  size 

696 

405 

443 

% 

% 

% 

Gender 

Male 

38.65 

30.37 

30.47 

Female 

61.35 

69.63 

69.53 

Age 

0-17 

18.00 

10.37 

9.71 

18-24 

4.93 

5.68 

6.55 

25-44 

36.75 

45.19 

44.47 

45-64 

27.16 

31.36 

31.83 

65-90 

13.18 

7.41 

7.45 

Race 

Hispanic 

14.80 

16.54 

16.03 

Black 

7.76 

7.90 

8.13 

Other 

77.44 

75.56 

75.85 

Education 

No  degree 

14.80 

17.28 

17.83 

HS/GED 

40.09 

42.72 

44.02 

Bachelor’s/Other 

19.40 

21.73 

20.99 

Advanced  degree 

7.76 

9.38 

8.80 

Under  16  Inapplicable 

17.96 

8.89 

8.35 

Insurance  coverage 

Private 

65.95 

62.20 

62.53 

Depression  Guideline 

Sample  at  subgroup  at  compliant 

MT5  visits  MT  5  visits  group 

Public 

Uninsured 

27.16  29.63  29.12 

6.90  8.15  8.35 

General  diagnosis 

Mental  health 

Medical  diagnosis  only 

No  diagnosis 

78.02  100.00  100.00 

16.52  na  na 

5.46  na  na 

Psychotropic  drug 

No 

Yes 

39.22  18.52  16.93 

60.78  81.48  83.07 

Health  status 

Excellent/very/good 

Fair/poor 

60.99  60.25  61.17 

39.01  39.25  38.83 

Mental  health 

Excellent/very/good 

Fair/poor 

54.59  51.60  53.05 

45.41  48.40  46.95 

Activity  limitation 

No 

Yes 

71.41  68.07  68.93 

28.59  31.93  31.07 

Social  limitations 

No 

Yes 

84.57  81.84  82.69 

15.43  18.16  17.31 

Appt  difficulty 

No 

Yes 

70.11  30.12  70.20 

29.89  69.88  29.80 

Phone  difficulty 

No 

Yes 

67.00  66.91  33.18 

33.00  33.09  66.82 

Provider  listens 

Yes 

No 

95.10  95.06  95.03 

4.90  4.94  4.97 

Provider  confidence 

Yes 

No 

93.52  93.83  93.68 

6.48  6.17  6.32 

Satisfied  with  provider/staff 

Satisfied 

Not  satisfied 

92.23  93.09  93.00 

6.77  6.91  7.00 

Satisfied  with  quality  of  care 

Satisfied 

Not  satisfied 

96.11  96.54  93.39 

3.89  3.46  3.61 
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In  support  of  the  third  model  estimating  differences  in  the  hazard  rate  between 
satisfied  and  not  satisfied,  changes  in  proportions  were  examined  and  between  more 
than  one  visit,  more  than  six  visits,  more  than  ten  visits  and  more  than  15  visits  (Table  3 
below). 


Table  3  Sample  differences  from  more  than  one  to  more  than  15  visits 


MT1  visit 

MT6  visits 

MT10  visits 

MT15  visits 

Sample  size 

1103 

640 

482 

357 

% 

% 

% 

% 

Gender 

Male 

37.62 

38.59 

39.00 

40.90 

Female 

62.38 

61.41 

31.00 

59.10 

Age 

0-17 

16.23 

17.99 

17.13 

17.96 

18-24 

5.53 

5.04 

4.28 

4.37 

25-44 

37.22 

36.69 

37.43 

37.38 

45-64 

27.20 

27.48 

26.44 

23.30 

65-90 

13.82 

12.81 

14.71 

16.99 

Race 

Hispanic 

14.51 

14.53 

14.52 

12.32 

Black 

8.52 

8.13 

8.09 

7.84 

Other 

76.97 

77.34 

77.39 

79.83 

Education 

No  degree 

17.32 

13.75 

13.90 

11.48 

HS/GED 

41.07 

40.16 

40.66 

39.78 

Bachelor’s/Other  degree 

19.49 

19.84 

19.09 

21.01 

Advanced  degree 

6.53 

8.13 

8.09 

8.12 

Under  16  Inapplicable 

15.59 

18.13 

18.26 

19.61 

Insurance  coverage 

Private 

66.91 

65.63 

62.66 

62.18 

Public 

26.38 

27.50 

29.88 

30.25 

Uninsured 

6.71 

6.88 

7.47 

7.56 

General  diagnosis 

Mental  health 

75.79 

79.53 

81.12 

81.23 

Medical  diagnosis  only 

16.68 

15.94 

15.77 

15.13 

No  diagnosis 

7.52 

4.53 

3.11 

3.64 

Psychotropic  drug 
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MT1  visit 

MT6  visits 

MT10  visits 

MT15  visits 

No 

42.97 

38.44 

34.85 

36.69 

Yes 

57.03 

61.56 

65.15 

63.31 

Health  status 

Excellent/very/good 

64.85 

61.58 

58.10 

56.55 

Fair/poor 

35.15 

38.42 

41.90 

43.45 

Mental  health 

Excellent/very/good 

60.19 

54.39 

50.84 

51.46 

Fair/poor 

39.81 

45.61 

49.46 

48.54 

Activity  limitation 

No 

73.46 

71.45 

69.62 

67.43 

Yes 

26.54 

28.55 

30.38 

32.57 

Social  limitations 

No 

85.50 

84.49 

83.16 

81.20 

Yes 

14.50 

15.51 

16.84 

18.80 

Appt  difficulty 

No 

68.54 

69.69 

69.50 

68.35 

Yes 

31.46 

30.31 

30.50 

31.65 

Phone  difficulty 

No 

93.64 

93.54 

94.08 

94.73 

Yes 

6.36 

6.46 

5.92 

5.27 

Provider  listens 

Yes 

94.73 

94.98 

94.38 

94.37 

No 

5.27 

5.02 

5.63 

5.63 

Provider  confidence 

Yes 

93.64 

93.42 

93.54 

94.08 

No 

6.36 

6.58 

6.46 

5.92 

Satisfied  with  provider/staff 

Satisfied 

94.01 

92.79 

92.08 

92.68 

Not  satisfied 

5.99 

7.21 

7.92 

7.32 

Satisfied  with  quality  of  care 

Satisfied 

95.37 

96.08 

95.42 

96.06 

Not  satisfied 

4.63 

3.92 

4.58 

3.94 

The  proportion  of  males  in  the  sample  increased  from  37%  having  more  than  one 
visit  to  41%  with  more  than  15  visits  (Table  3).  In  the  treatment  guideline  compliant 
group  the  proportion  of  males  was  even  lower  at  30%  with  the  difference  attributed  to 
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selection  into  the  depression  subgroup  (Table  2).  The  highest  proportion  of  females  is 
seen  in  the  population  having  more  than  one  visit  (Table  1),  those  selected  into  the 
depression  subgroup  (Table  2),  and  among  individuals  treatment  guideline  compliant 
(69%)  (Table  2). 

The  mean  years  of  education  at  each  visit  did  not  vary4.  However,  there  were  changes 
in  proportions  by  education.  The  number  of  respondents  with  less  than  a  high  school 
education  decreased  from  17.3%  at  more  than  one  visit  to  11.5%  at  more  than  15  visits 
(Table  3).  The  proportion  of  respondents  with  less  than  high  school  was  higher  for  the 
compliant  group  at  17.8%  with  most  of  the  change  due  to  selection  in  the  depression 
subgroup  (Table  2).  Respondents  with  a  high  school  education  decreased  from  41%  to 
39.8%  (Table  3),  but  the  proportion  in  the  compliant  group  was  highest  at  44%  with  the 
difference  due  both  to  selection  in  the  depression  subgroup  (42.7%)  and  having  more 
than  five  visits  (40.1%)  (Table  2).  The  proportion  of  individuals  with  college  increased 
between  more  than  one  and  more  than  15  visits  (Table  3).  Respondents  with  a 
bachelor’s  degree  increased  from  19.5%  to  21%  with  most  of  the  increase  occurring 
between  10  and  15  visits  (Table  3).  In  the  compliant  group  the  proportion  with  a 
bachelor’s  degree  did  not  increase  (Table  2).  Individuals  with  an  advanced  degree 
increased  from  6.5%  to  8%  with  most  of  the  increase  occurring  between  6  and  15  visits 
(Table  3).  In  the  compliant  group  the  proportion  with  an  advanced  degree  was  8.8% 
which  was  less  than  for  the  proportion  in  the  depression  subgroup  at  9.4%  but  greater 
than  the  total  sample  at  more  than  five  visits  (Table  2). 

4  Mean  years  of  education  at  more  than  one  visit  was  1 1. 4(4.4)  and  by  more  than  15  visits  was  still 
1 1 .7(4.5).  The  mean  was  12.2(3.8)  for  compliance  subgroup  and  12.2(3.8)  at  more  than  5  visits. 
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There  were  changes  in  sample  proportions  by  insurance  between  one  and  15  visits 
(Table  3)  and  between  having  more  than  five  visits  and  treatment  guideline  compliance 
(Table  2).  The  proportion  of  privately  insured  decreased  from  67%  of  the  population  at 
more  than  one  visit  to  62%  by  more  than  15  visits  (Table  3).  In  the  compliant  group  the 
privately  insured  made  up  62.5%  with  most  of  the  difference  due  to  selection  in  the 
depression  subgroup  (Table  2).  The  proportion  of  publicly  insured  increased  from  26% 
at  more  than  one  visit  to  30%  at  more  than  1 5  visits  with  most  of  the  increase  occurring 
between  6  and  15  visits  (Table  3).  The  uninsured  represented  6.7%  at  more  than  one 
visit  to  7.5%  at  more  than  15  visits  (Table  3).  The  proportion  in  the  compliant  group 
was  slightly  higher  at  8.35  with  the  difference  attributed  to  selection  into  the  depression 
subgroup  (Table  2).  The  proportion  of  individuals  on  a  psychotropic  drug  increased 
gradually  from  57%  to  63%  over  15  visits  (Table  3).  The  proportion  of  respondents  in 
the  compliant  group  was  83%  with  81.5%  due  to  selection  into  the  depression  subgroup 
(Table  2). 

There  were  differences  in  the  population  by  health  and  mental  health  variables  from 
one  to  three  visits  (Table  1),  between  more  than  five  visits  and  the  treatment  compliant 
group  (Table  2)  and  between  one  and  15  visits  (Table  3).  The  proportion  of  individuals 
with  a  mental  health  diagnosis  increased  gradually  from  76%  at  more  than  one  visit  to 
81%  at  more  than  15  visits.  The  proportion  of  individuals  reporting  poor  health  status 
increased  from  35%  at  more  than  one  visit  and  37%  at  more  than  three  visits  to  43.5%  at 
more  than  15  visits.  In  the  compliant  group  the  percentage  reporting  poor  health  status 
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was  38%  (Table  2).  The  proportion  of  individuals  reporting  poor  mental  health  status 
increased  from  almost  40%  at  more  than  one  visit  and  43%  at  more  than  three  visits  to 
48.5%  at  more  than  15  visits  (Table  3).  In  the  compliant  group  the  percentage  reporting 
poor  mental  health  was  47%  (Table  2).  The  proportion  reporting  activity  limitations  and 
social  limitations  increased  between  more  than  one  and  more  than  15  visits  (Table  3). 
Individuals  reporting  activity  limitations  increased  from  26.5%  at  more  than  one  visit  to 
32.5%  at  more  than  15  visits  (Table  3);  31%  for  the  treatment  guideline  compliant  group 
(Table  2).  The  proportion  of  individuals  reporting  social  limitations  increased  from 
14.5%  at  more  than  one  visit  to  18.8%  at  more  than  15  visits  (Table  3);  17.3%  in  the 
treatment  guideline  compliant  group  (Table  2). 

There  were  no  changes  in  sample  proportions  for  the  satisfaction  variables.  The 
proportion  of  respondents  reporting  dissatisfaction  ranged  from  3-8%  across  all  visits. 
The  proportion  of  the  population  reporting  satisfaction  with  quality  of  care,  confidence 
in  provider  and  no  phone  difficulty  increased  between  more  than  one  and  more  than  15 
visits.  The  proportion  reporting  satisfaction  with  provider/staff  decreased  between  more 
than  one  and  more  than  15  visits.  There  was  no  change  in  the  proportion  reporting 
appointment  difficulty  and  provider  listens.  In  summary  significant  changes  in 
proportions  were  found  between  more  than  one  and  more  than  three  visits  for 
education,  diagnosis,  psychotropic  drug,  health  and  mental  health  status,  activity  and 
social  limitations. 

There  were  changes  in  proportions  between  all  respondents  with  more  than  five 
visits,  respondents  with  a  diagnosis  of  depression  and  more  than  five  visits,  and 
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respondents  with  a  diagnosis  of  depression  who  were  treatment  guideline  compliant  for 
education,  psychotropic  drug,  mental  health,  activity  and  social  limitations.  There  were 
differences  in  proportions  between  respondents  with  more  than  one  visit  and  more  than 
15  visits  for  education,  insurance  coverage,  diagnosis,  psychotropic  drug,  health  and 
mental  health  status,  activity  and  social  limitations.5  Consistently  across  all  comparisons 
lower  education  levels,  and  better  health  and  better  mental  health  status  yielded  higher 
rates  of  dropout  while  higher  education,  mental  health  diagnosis,  psychotropic  drug, 
poor  health  and  mental  health  status,  and  greater  activity  and  social  limitations  yielded 
lower  dropout  rates.  The  next  section  proceeds  with  the  results  by  describing  bivariate 
associations  between  variables  through  chi  square  tests  of  association  and  univariate 
regression  models. 

Bivariate  associations 

Two  way  contingency  tables  and  chi-square  statistics  were  used  to  examine  bivariate 
associations  between  predictor  variables,  covariates  and  the  probability  of  more  than  one 
visit,  more  than  two  visits  and  more  than  three  visits.  The  purpose  was  to  confirm 
variables  for  selection  into  a  logistic  regression  model.  Pearson  chi-square  procedures 
were  used  with  a  p  value  set  at  .25  as  recommended  by  (Hosmer  &  Lemeshow,  2000). 
As  shown  in  Table  4  and  Table  5  covariates  related  to  impairment  showed  the  strongest 
bivariate  association  to  having  subsequent  visits.  Sociodemographic  variables  gender  and 
insurance  were  not  highly  significant  across  visits  except  at  the  variable  screening  level  of 
p<.25.  There  was  a  significant  association  between  age  and  having  greater  than  one  visit, 

5  Analysis  of  variance  was  used  in  a  separate  analysis  comparing  the  population  at  exactiy  one  visit,  exactly 
two,  exacdy  three  visits  and  exactly  one  visit,  exactly  five  visits,  exacdy  ten  visits  and  exacdy  1 5  visits. 

There  were  no  statistically  significant  differences  in  the  population  except  for  differences  in  education 
[F(l,1054)=3.77(p=  05)]  and  mental  health  [F(l , 1 054) = 1 6 .23 (p < . 000 1 )] . 
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two  visits  and  three  visits.  There  was  a  significant  association  between  education  and 
having  greater  than  two  visits  and  greater  than  three  visits.  None  of  the  six  satisfaction 
variables  were  associated  with  the  probability  of  a  subsequent  visit.  At  this  stage  all  the 
selected  variables  were  retained. 


Table  4  Bivariate  analysis  of  the  first  four  visits 


Pearson  Chi-square  (df) 

GE  1  visits 

GE  2  visits 

GE  3  visits 

Age 

12.90  (4)  ** 

18.39(4)  *** 

22.91(4)  *** 

Gender 

1.55  (1)  * 

1.16(1) 

0.04(1) 

Education 

5.29  (4) 

8.78(4)  ** 

9.02(4)  ** 

Race 

0.46  (2) 

0.41(2) 

0.81(2) 

Insurance  changes 

0.48  (3) 

3.08(3) 

4.96(3)* 

Insurance  coverage 

4.71(2)  * 

3.77(2)  * 

2.61(2) 

Diagnosis 

56.39  (2)  *** 

55.17  (2)  *** 

48.0  (2)  *** 

Psychotropic  drug 

49.42  (1)  *** 

51.65(1)  *** 

48.22(1)  *** 

Antidepressant  drug 

43.59  (1)  *** 

54.43  (1)  *** 

47.67  (1)  *** 

Health  status 

14.90** 

14.12** 

13.08*** 

Mental  health  status 

55.90*** 

65.32*** 

74.00*** 

Any  limitation 

25.77  (1)  *** 

28.00  (1)  *** 

18.49  (1)  *** 

Social  limitation 

5.80  (1)  * 

6.07  (1)  * 

5.42  (1)  * 

Satisfaction  w/ staff 

1.24  (3) 

1.54(3) 

1.92  (3) 

Satisfaction  w/ quality 

4.62  (3) 

1.31  (3) 

0.53  (3) 

Appt  difficulty 

3.45  (3) 

1.65  (3) 

0.94  (3) 

Phone  difficulty 

2.08  (3) 

2.30  (3) 

2.90  (3) 

Confident  in  provider 

0.05  (1) 

0.001  (1) 

0.03  (1) 

Provider  listens 

0.93  (1) 

0.32  (1) 

0.12  (1) 

*p<.25  **p<,05  ***p<.01 
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Table  5  Bivariate  analysis  of  depression  treatment  compliance 


Pearson  Chi-square  (df) 

Compliant  if  >5  visits  or  >  3  antidepressant  Rx 

Gender 

0.40  (1) 

Age 

12.43  (4)  * 

Education 

6.73  (4)  * 

Race 

2.68  (2) 

Insurance  changes 

1.25  (3) 

Insurance  coverage 

8.55  (2) 

Truncated  data  indicator 

15.76  (1)  *** 

Health  status 

16.05  (4)  *** 

Mental  health  status 

28.51  (4)  *** 

Any  limitation 

6.70  (1)  *** 

Social  limitation 

.04  (1) 

Satisfaction  w/ staff 

4.31  (3)  * 

Satisfaction  w/ quality 

5.91  (3)  * 

Appt  difficulty 

1.25  (3) 

Phone  difficulty 

1.07  (3) 

Confident  in  provider 

1.66  (1)* 

Provider  listens 

1.74  (1)  * 

*p<.25  **p<.05***p<.01 


Confounding  effects 

Selected  covariates  (age,  sex,  education,  race,  impairment,  insurance)  have  been 
associated  with  number  of  visits,  satisfaction  with  care  or  both.  Variables  associated  with 
both  a  predictor  and  an  outcome  variable  can  distort  any  statistical  relationship  if  not 
controlled.  Covariates  were  evaluated  for  confounding  effects  on  the  relationship 
between  satisfaction  and  number  of  visits.  Each  covariate  was  entered  separately  into 
three  different  sets  of  univariate  regression  equations.  The  first  regressing  satisfaction 
with  care  on  each  covariate  in  separate  logistic  models.  The  second  regressing  the 
probability  of  more  than  one  visit  on  each  covariate  in  separate  logistic  models.  The 
third  regressing  the  number  of  visits  on  each  covariate  in  linear  models  since  hazard 


models  will  consider  the  total  number  of  visits.  Because  of  the  skewed  distribution  of 
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number  of  mental  health  visits  a  natural  logaritm  transformation  for  visits  was  used.  The 
results  are  described  below. 

Being  female  was  significandy  related  to  high  satisfaction  with  provider/staff  and 
quality  of  care  but  was  not  related  to  number  of  visits.  Race  was  related  to  appointment 
difficulty.  African  American  respondents  were  almost  twice  more  likely  than  Hispanic 
respondents  to  report  no  appointment  difficulty.  White  respondents  were  also  more 
likely  than  Hispanics  to  report  no  appointment  difficulty.  Race  was  not  significandy 
related  to  number  of  visits.  Age  was  related  to  satisfaction.  Respondents  age  18-24  were 
50%  less  likely  to  be  highly  satisfied  with  provider/staff  (p<.01)  and  were  40%  less  likely 
to  have  more  than  one  mental  health  visit  (p<.05).  Respondents  65-90  were  1.5  times  as 
likely  to  report  no  appointment  difficulty  (p<.05)  and  were  40%  less  likely  to  have  more 
than  one  mental  health  visit  (p<.05).  Age  is  a  confounding  variable  for  appointment 
difficulty  and  the  probability  of  a  subsequent  visit. 

Respondents  with  an  advanced  degree  were  50%  less  likely  to  report  no  appointment 
difficulty  (p<.001),  but  having  an  advanced  degree  did  not  significandy  affect  the 
probability  of  more  than  one  visit.  Education  had  a  significant  positive  relationship  to 
number  of  visits  with  higher  education  associated  with  a  greater  number  of  visits.  For 
example,  individuals  with  an  advanced  degree  (Master’s /doctorate)  had  an  average  of  19 
visits — a  higher  average  than  even  individuals  with  poor  mental  health  (Appendix  A 
Table  22).  Education  is  a  potential  confounder  for  modeling  the  relationship  between 
appointment  difficulty  and  the  probability  of  a  second  and  third  visit.  Uninsured 
respondents  were  only  half  as  likely  as  privately  insured  respondents  to  be  highly 
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satisfied  with  provider/staff;  but  having  no  insurance  was  related  to  a  greater  number  of 
visits.  Type  of  insurance  had  a  statistically  significant  effect  on  visits  with  private 
insurance  being  associated  with  fewer  visits  (  p<.001).  Table  22  (Appendix  A)  shows 
respondents  with  private  insurance  having  an  average  of  12  visits,  uninsured  had  an 
average  of  13  visits  and  publicly  insured  had  an  average  of  15  visits.  The  type  of 
insurance  coverage  may  be  a  confounding  variable.  The  number  of  survey  period 
months  covered  by  some  type  of  insurance  was  not  a  significant  predictor  of  satisfaction 
or  number  of  visits. 

Among  the  impairment  variables,  respondents  with  a  medical  diagnosis  only  were 
45%  less  likely  to  have  confidence  in  the  provider  and  respondents  with  no  diagnosis 
were  50%  less  likely  to  have  confidence  in  the  provider.  Diagnosis  was  a  predictor  of  the 
number  of  visits.  Respondents  with  a  medical  diagnosis  were  55%  less  likely  to  have 
more  than  one  visit  and  those  with  no  diagnosis  were  60%  less  likely  to  have  more  than 
one  visit.  Mental  health  diagnosis  was  also  a  significant  predictor  of  overall  number  of 
visits.  Diagnosis  is  a  confounding  variable.  Health  and  mental  health  were  significant 
predictors  of  having  more  than  one  visit  and  for  number  of  visits,  but  neither  were 
related  to  satisfaction  with  care.  Respondents  with  activity  limitations  were  45%  less 
likely  to  be  satisfied  with  quality  of  care  and  30%  less  likely  to  report  no  appointment 
difficulty  but  were  over  two  times  more  likely  to  have  more  than  one  visit.  Having 
activity  limitations  was  also  associated  with  a  greater  number  of  visits.  Social  limitations 
was  not  associated  with  satisfaction  with  care,  but  was  associated  with  both  having  more 
than  one  visit  and  having  a  greater  number  of  visits  overall. 
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Summary 

This  section  has  examined  bivariate  association  in  the  data.  First,  I  described  general 
trends  in  number  of  visits  for  each  predictor  and  covariate  finding  that  individuals  age 
65-90,  with  less  education,  in  excellent  mental  health  and  not  satisfied  with  quality  of 
care  had  the  fewest  visits  while  age  0-17,  with  an  advanced  degree,  publicly  insured,  in 
excellent  health,  poor  mental  health  and  not  satisfied  with  provider/ staff  had  the  greatest 
number  of  visits.  Second,  I  described  changes  in  the  population  occurring  with  each 
visit  interval  as  respondents  discontinued  care.  This  revealed  significant  changes  in  the 
population  by  education,  diagnosis,  health  and  mental  health,  psychotropic  drugs, 
activity  and  social  limitations  and  insurance  coverage. 

Third,  to  clarify  variable  selection  for  model  building,  I  used  contingency  tables  for 
each  predictor  and  covariate  to  compare  proportions  in  each  cell  for  more  than  one, 
more  than  two  and  more  than  three  visits.  Chi-square  statistics  were  used  to  evaluate  the 
statistical  likelihood  of  obtaining  cell  size  at  random.  This  revealed  significant 
associations  between  the  probability  of  more  than  two,  three  and  four  visits  and  age, 
gender,  diagnosis,  psychotropic  drug,  health  and  mental  health  status,  insurance  coverage 
and  activity  and  social  limitations.  Fourth,  in  this  section  the  covariates  were  examined 
for  potential  confounding  effects  on  the  relationship  between  satisfaction  with  care  and 
number  of  visits  and  several  potential  confounders  were  identified:  age,  education, 
insurance,  diagnosis  and  activity  limitations.  The  next  section  will  proceed  with 
multivariate  analysis.  The  six  predictor  variables  and  covariates  will  be  evaluated  for  their 
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contribution  to  logistic  models  estimating  the  probability  of  more  than  one,  more  than 
two  and  more  than  three  visits,  and  the  probability  of  depression  treatment  compliance. 

Multivariate  analysis 

Multivariate  methods  were  used  to  test  the  three  research  hypothesis.  First  logistic 
regression  models  were  estimated  for  the  probability  of  a  more  than  one,  two  and  three 
visits.  Second  a  logistic  regression  model  was  estimated  for  the  probability  of  depression 
treatment  compliance  using  a  depression  subgroup.  Third  survival  analysis  was  used  to 
estimated  differences  in  time-to-failure  rates  between  satisfied  and  not  satisfied 
respondents.  Model  specifications,  i.e.  coding,  choice  of  covariates  and  variable  ordering 
for  the  logistic  model  predicting  more  than  one  visit  served  as  a  base  model  from  which 
subsequent  models  were  derived.  The  purpose  was  to  keep  the  models  as  comparable  as 
possible  to  support  comparisons  across  models.  Visit  weights  constructed  as  the  ratio  of 
mental  health  to  total  health  care  visits,  were  added  as  importance  weights  to  each 
regression  model.  These  serve  to  minimize  the  influence  of  satisfaction  evaluations  based 
on  health  not  mental  health  care  experiences. 

With  the  exception  of  the  survival  model,  each  of  the  three  main  effects  models  were 
estimated  both  with  and  without  MEPS  analytic  weights.  To  remind  the  reader  analytic 
weights  adjust  for  the  complex  survey  design  features  inherent  to  large  data  sets.  In 
complex  survey  design  the  probabilities  of  various  subjects  being  selected  for  inclusion 
into  the  study  are  not  equal  because  of  oversampling,  stratification  and  nonresponse 
adjustment.  This  can  affect  the  accuracy  of  statistical  analyses  results  and  increase  the 
likelihood  of  a  Type  1  error  (S.  B.  Cohen,  2000;  Kneipp  &  Yarandi,  2002).  Variance 
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estimation  weights  adjust  for  the  increased  variance  produced  due  to  unequal  selection 
probabilities,  nonresponse  and  stratification.  Sampling  weights  adjust  sample  estimates 
to  reflect  the  total  proportion  of  people  in  the  U.S.  population.  Both  weighted  and  un¬ 
weighted  models  were  used  because  each  carries  some  inherent  risks  to  both.  Without 
the  analytic  weights  the  data  there  is  no  adjustment  for  the  oversampling  of  Africa 
Americans  and  Hispanics  and  adjustment  for  survey  nonresponse.  With  the  weights  the 
standard  errors  become  inflated  when  cell  size  becomes  small  as  it  invariably  will  in  some 
models. 


Satisfaction  with  care  and  the  probability  of  a  subsequent  visit 

The  first  logistic  regression  models  assessed  the  effect  of  satisfaction  with  care  on  the 
probability  of  a  subsequent  visit  after  adjusting  for  sociodemographic  and  impairment 
factors.  Models  were  estimated  for  the  probability  of  more  than  one  visit,  more  than  two 
visits  and  more  than  three  visits.  The  procedures  for  model  selection  are  reported  below. 
Results  for  each  model  are  discussed  separately  for  satisfaction  with  quality  of  care, 
satisfaction  with  provider/ staff,  confidence  in  provider,  provider  listens,  appointment 
and  telephone  difficulty.  A  summary  of  the  findings  is  presented  at  the  end  of  this 
section  before  proceeding  to  the  next  section. 

A  logistic  regression  model  estimating  the  probability  of  more  than  one  visit  was  fit 
for  main  effects.  Although  there  were  empirical  reasons  for  including  all  the  covariates, 
the  model  was  still  tested  to  evaluate  the  best  set  of  covariates.  Forward  selection 
procedures  were  used  to  identify  variables  that  did  not  contribute  significantly  to  the 
model.  Hosmer  &  Lemeshow  (2000)  recommend  the  likelihood  ratio  test  (LRT)  which 
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compares  the  differences  in  the  log  likelihood  function  at  each  step  of  model  estimation 
to  test  whether  additional  variable  contribute  significantly  to  the  analytic  model.  A  best 
subsets  method  was  used  whereby  the  sociodemographic  variables  were  entered 
simultaneously  followed  by  impairment  variables  and  satisfaction  with  care  variables. 
Using  this  method  sociodemographic  and  impairment  variables  contributed  significantly 
to  the  model  at  p<001.  The  satisfaction  variables  made  a  marginally  significant 
contribution  at  p=.09.  To  evaluate  the  relative  contribution  of  each  variable  to  the  three 
models,  regression  coefficients  were  standardized  (Table  24  in  Appendix  B).  This  reveals 
how  much  variance  in  outcomes  was  uniquely  contributed  by  each  variable.  Satisfaction 
with  quality  of  care  contributed  almost  six  percent  of  unique  variance  in  Model  1  (p<.01) 
and  confidence  in  provider  contributed  almost  four  percent  of  the  variance  in  Model  1 
(p<.05).  Activity  limitations,  health  and  mental  health  contributed  the  greatest  portion  of 
unique  variance  (p<.05).  The  unique  variance  contributed  by  gender  and  education  was 
each  made  up  six  percent  of  the  variance. 

The  three  subsets  of  variables  made  a  significant  contribution  to  the  model  and  were 
retained  in  the  model  estimating  more  than  one  visit.  The  same  model  specifications 
were  used  to  estimate  the  probability  more  than  two  and  more  than  three  visits.  Table  6 
below  shows  the  main  effects  for  the  relationship  between  satisfaction  with  care  and 
probability  of  more  than  one  visit.  The  model  was  significant  at  p<.0001  and  accounted 
for  27%  of  the  variance  in  the  probability  of  having  more  than  one  visit.  The  second 
model  (not  shown)  estimating  the  probability  of  more  than  two  visits  was  significant 
p<.0001  accounting  for  26%  of  the  variance  in  the  probability  of  having  more  than  two 
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visits.  The  third  model  (not  shown)  was  significant  p<.0001  and  accounted  for  27%  of 
the  variance  in  probability  of  having  more  than  three  visits. 


Table  6  Main  effects  model  of  the  relationship  between  satisfaction  with  care  and  the  probability 
of  more  than  one  visit 


Ref  group  in  italics 

LR  X2  statistic  (df=36) 

OR  (SE) 

Coefficient 

95%  Cl 

Sex 

Male 

Female 

1.63(.28)** 

.49 

1.17-2.27 

Race 

White 

Hispanic 

.90(.21) 

-.10 

.54-1.42 

Black 

.58(36)* 

-.54 

.34-1.0 

Age 

25-44 

0-17 

8.29(10) 

2.12 

.77-89.1 

18-24 

,98(.32) 

-.02 

.52-1.84 

45-64 

1.13(.22) 

.13 

.78-1.66 

65-90 

1.27(33) 

.18 

.77-2.11 

Education 

No  degree 

High  school/GED 

1.19(.28) 

.18 

.75-1.9 

Bachelor’s/other 

1.94(.57)* 

.67 

1.1-3.44 

Master’s/Doctorate 

2.2(.91) 

.79 

.98-4.94 

Under  16  Inappl 

.20(.24) 

-1.60 

.02-2.1 

Insurance 

Private 

Public 

,95(.20) 

-.05 

.63-1.44 

Uninsured 

.42(32) 

-.88 

.09-1.84 

Ins  coverage 

18-24  months 

12-17  months 

1.17(.45) 

.14 

.54-2.47 

6-11  months 

1.36(.65) 

31 

.54-3.45 

0-6  months 

1. 4(1.0) 

.33 

32-5.95 

Diagnosis 

Mental  health 

Medical  only 

.78(36) 

-.25 

.51-1.18 

No  diagnosis 

.81(.22) 

-.22 

.48-136 

Health 

Excellent 

Veiy  good 

.92(.24) 

-.09 

.55-1.52 

1.25(33) 

.23 

.75-2.11 

Fair 

1.71  (.52) 

.54 

.94-3.12 

Poor 

2.33(.90)* 

.84 

1.1-4.95 

Mental  health 

Excellent 

Very  good 

1.82(.47)* 

.60 

1.09-3.02 

Good 

1.98(.51)** 

.68 

1.2-3.26 

Fair 

1.8(.51)* 

.57 

.99-3.12 
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Ref  group  in  italics 

LR  X2  statistic  (df=36) 

OR  (SE) 

Coefficient 

95%  Cl 

Poor 

1.55(.60) 

.44 

.72-3.29 

Activity  limits 

No 

Yes 

2.58(.69)** 

.95 

1.52-4.37 

Social  limits 

No 

Yes 

,89(.27) 

-.12 

.49-1.6 

Psychotropic 

No 

Yes 

1.49(.27)* 

.40 

1.04-2.12 

Appt  difficulty 

Very  difficult 

No  difficulty 

.97(.18) 

-.03 

.67-1.39 

Phone  difficulty 

Very  difficult 

No  difficulty 

.90(.  16) 

-.12 

.62-1.27 

Provider  listens 

No 

Yes 

.85(.40) 

-.16 

.34-2.13 

Confident  in 

Provider 

No 

Yes 

2.16(.90) 

.77 

.96-4.87 

Satisfaction  with 
provider/staff 

Not  satisfied 

Satisfied 

2.10(.79)* 

.74 

1.0- 1.4 

Satisfaction  w 
quality  of  care 

Not  satisfied 

Satisfied 

.23(.14)* 

-1.48 

.07-.74 

Model  R2  =  27 

Probability  of  obtaining  a  X2  statistic  at  least  as  large  at  random:  *p<.05  **p<.01 


The  satisfaction  variables  that  will  be  discussed  below  did  not  emerge  as  strong 
predictors  of  the  probability  of  more  than  one  visit.  However  a  relationship  is  sometimes 
not  clearly  evident  if  it  changes  over  levels  of  a  third  variable.  To  assess  for  interaction 
effects  multiplicative  terms  were  created  for  each  satisfaction  variable  and  each  level  of 
gender,  race,  education,  insurance,  health  and  mental  health  since  these  could  moderate 
the  relationship  between  satisfaction  and  the  probability  of  a  subsequent  visits.  The 
terms  were  entered  individually  into  each  of  the  three  main  effects  models.  An 
interaction  is  present  if  the  coefficient  of  the  interaction  term  is  statistically  significant.  A 
significant  interaction  term  indicates  a  difference  in  the  effect  of  that  level  of  the  dummy 
variable  compared  to  the  omitted  reference  group.  The  following  sections  address  the 
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main  effects  and  significant  interactions  found  for  each  of  the  six  satisfaction  variables  in 
models  predicting  more  than  one  visit,  more  than  two  visits  and  more  than  three  visits. 
Differences  between  un-weighted  and  weighted  models  is  addressed. 

Satisfaction  with  quality  of  care 

The  main  effects  model  for  the  probability  of  more  than  one  visit  indicates 
satisfaction  with  quality  of  care  is  a  significant  predictor  (Table  6).  The  direction, 
consistent  in  both  weighted  and  un-weighted  models,  was  unexpected.  Individuals  who 
report  high  satisfaction  with  quality  of  care  were  80%  less  likely  to  have  more  than  one 
visit  (p<.05).  Satisfaction  with  quality  of  care  was  not  significant  for  the  probability  of 
more  than  two  visits  or  more  than  three  visits.  Interaction  terms  indicate  the  relationship 
between  satisfaction  with  quality  of  care  and  the  probability  of  a  subsequent  visit  may  be 
moderated  by  race,  education,  age  and  diagnosis.  This  means  that  for  one  or  more  levels 
of  each  moderating  variable,  satisfaction  with  quality  of  care  may  have  a  negative  (smaller 
or  weaker)  or  positive  (larger  or  stronger)  effect  on  the  probability  of  a  subsequent  visit 
compared  to  the  other  levels  of  the  same  variable. 

Table  7  shows  interaction  terms  that  were  significant  in  both  un-weighted  and  weighted 
interaction  models.  The  relationship  between  satisfaction  with  quality  of  care  and  the 
probability  of  more  than  one  visit  was  93%  weaker  for  African  Americans  as  compared 
to  white  and  Hispanic  groups  and  61%  stronger  for  females  compared  to  males.  The 
effect  of  satisfaction  with  care  on  the  probability  of  a  subsequent  visit  is  96%  weaker  for 
individuals  with  a  high  school  education  and  considerably  stronger  for  individuals  with 
college.  The  effect  of  diagnosis  on  the  strength  of  the  relationship  between  satisfaction 
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with  quality  of  care  and  the  probability  of  a  subsequent  visit  is  33%  larger  among 
individuals  with  a  mental  health  diagnosis  compared  to  individuals  with  a  medical 
diagnosis  only  or  no  diagnosis.  It  was  83%  weaker  for  individuals  having  a  medical 
diagnosis  only.  The  effect  of  health  on  satisfaction  with  quality  of  care  was  98%  weaker 
for  individuals  reporting  excellent  or  very  good  health  compared  to  respondents 
reporting  poor  health. 


Table  7  Significant  interactions  with  satisfaction  with  quality  of  care  in  weighted  and  un¬ 
interaction  weighted  models 


Main  effects 

Significant 
interaction  effects 

%  Net  effect  of  satisfaction  with  quality  of  care  on 
probability  of  subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

Un-weighted  analytic  models 

Sex 

Female 

0.61* 

Race 

Black 

-0.93* 

Age 

25-44 

Education 

High  School 

-0.96* 

College 

0.19* 

Weighted  analytic  models a 

Race 

Black 

-0.96* 

Age 

25-44 

-0.90* 

Education 

High  school 

-0.92t 

College 

2.10** 

1.62t 

Diagnosis 

Mental  health 

0.05* 

0.33* 

Medical  only 

-0.83* 

Health  status 

Excel/very  good 

-0.98** 

*  p<  05  **  p<  01  ^approached  p<  05 8  variance  estimation  and  sampling  weights 


Satisfaction  with  quality  of  care  seems  to  be  a  significant  predictor  of  more  than  one 
visit,  but  the  direction  of  the  relationship  was  unexpected.  The  direction  suggests  being 
highly  satisfied  with  the  quality  of  care  decreases  the  probability  of  a  subsequent  visit. 
One  interpretation  is  that  being  satisfied  with  quality  of  care  after  the  first  visit  equates 
to  a  individuals  having  obtained  a  desired  outcome  and  no  longer  needing  the  service. 
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This  lends  support  for  the  notion  that  individuals  who  only  have  one  visit  are  indeed 
satisfied  with  the  available  outcomes  for  information  and  reduced  uncertainty 
independent  of  treatment.  Another  possibility  is  that  the  variable  is  picking  up  the 
effects  of  the  health  care  portion  of  the  health  to  mental  health  utilization  ratio.  The 
cases  are  weighted  towards  individuals  with  less  complex  medical  problems  for  which 
easily  satisfied  outcomes  are  more  obtainable  after  one  visit.  This  was  tested  by 
estimating  the  model  without  the  visit  weights.  The  change  had  no  effect  on  the 
direction  or  significance  of  satisfaction  with  quality  of  care.  In  addition  to  the  negative 
relationship,  interaction  models  suggests  the  extent  to  which  satisfaction  with  quality  of 
care  is  an  important  predictor  of  a  subsequent  visit  varies  by  race,  education,  diagnosis 
and  health  status. 

Satisfaction  with  provider/staff 

The  main  effects  model  indicates  satisfaction  with  provider/staff  is  a  significant 
predictor  of  more  than  one  visit  with  individuals  highly  satisfied  with  provider/staff 
being  more  than  twice  as  likely  to  have  more  than  one  visit  (p<.05)  (Table  6).  In  the 
weighted  model  individuals  satisfied  with  provider/ staff  were  two  times  as  likely  to  have 
more  than  two  visits  (p<.05)(not  shown).  Interaction  models  indicate  the  relationship 
may  be  moderated  by  age,  education,  health  and  insurance  coverage  (Table  8).  When  age 
is  25-44  the  effect  of  satisfaction  with  provider/staff  on  the  probability  of  more  than 
one  visit  is  19%  weaker  than  for  other  age  groups.  The  main  effects  model  stratified  by 
age  in  a  separate  analysis  (Table  27  Appendix  B)  suggests  individuals  age  25-44  satisfied 
with  provider/ staff  are  50%  less  likely  to  have  more  than  two  visits  while  individuals  age 
45-64  are  over  six  times  more  likely  to  have  more  than  two  visits.  The  effect  of 
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satisfaction  with  provider/ staff  on  the  probability  of  more  than  two  and  more  than  three 
visits  is  47%  larger  for  individuals  with  a  mental  health  diagnosis  and  for  excellent/ very 
good  health  in  predicting  more  than  three  visits.  The  effect  of  satisfaction  with 
provider/staff  on  a  subsequent  visit  was  stronger  for  individuals  with  an  advanced 
degree  and  individual  sin  excellent/ very  good  health.  The  effect  was  71%  weaker  among 
privately  insured  individuals.  In  summary  satisfaction  with  provider/staff  is  a  positive 
predictor  of  more  than  one  visit  and  more  than  two  visits  with  significant  differences  in 
effect  for  age,  diagnosis,  health  and  even  insurance  across  all  three  models. 


Table  8  Significant  interaction  effects  with  satisfaction  with  provider/staff  care  in  weighted  and 
un-interaction  weighted  models 


Significant 

Main  effects  interaction  effects 

%  Net  effect  of  satisfaction  with  provider/staff  on 
probability  of  subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

UNWEIGHTED 

Diagnosis  cat. 

Mental  health 

1.35t 

0.47* 

Health  status 

1.88* 

WEIGHTED a 

Age 

-0.28* 

-0.1 9f 

Education 

22.08* 

Health  status 

6.99* 

Insurance  coverage 

Privately  insured 

-0.71* 

*  p<.05  **  p<  01  Approached  p<.05  a  variance  estimation  and  sampling  weights 


Provider  listens 

Provider  listens  was  not  a  significant  predictor  of  a  subsequent  visit  in  un-weighted  or 
weighted  main  effects  models  (Table  6).  Several  interaction  terms  were  significant.  Table 
9  shows  interactions  between  provider  listens  and  education,  diagnosis  and  health.  The 
effect  of  whether  the  provider  listens  on  the  probability  of  a  subsequent  visit  was  greater 
for  individuals  with  college  but  was  78%  weaker  for  individuals  with  a  high  school 
education.  Provider  listens  had  65%  less  of  an  effect  on  the  probability  of  a  subsequent 
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visit  for  individuals  with  a  medical  diagnosis  than  individuals  with  a  mental  health 
diagnosis  and  84-90%  less  of  an  effect  for  individuals  in  excellent/ very  good  compared 
to  individuals  in  poor  health.  Even  though  provider  listens  was  not  significant  in  main 
effects  models,  interaction  terms  are  largely  consistent  with  the  pattern  of  interactions 
found  with  the  two  previous  satisfaction  variables. 


Table  9  Significant  interaction  effects  with  provider  listens  in  weighted  and  un-weighted  models 


Main  effects 

Significant 
interaction  effects 

%  Net  effect  of  provider  listens  on  probability  of 
subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

UNWEIGHTED 

Education 

College 

2.55f 

Excel/very  good 

-0.84* 

WEIGHTED a 

Education 

High  school 

-0.78* 

College 

4/57* 

is  yjl  BSf  dl!  ■»•!  Jl 

nfliimiw 

Health  status 

Excel/very  good 

-0.90** 

*  p<  05  **  p<  01  Approached  p<  05  a  variance  estimation  and  sampling  weights 


Confidence  in  provider 

Confidence  in  provider  was  a  positive  predictor  of  more  than  one  visit  in  the 
weighted  model  (p<.05)  (Table  25  in  Appendix  B)  and  approached  significance  in  the 
un-weighted  model  (Table  6).  Individuals  with  confidence  in  the  provider  were  over 
twice  as  likely  to  have  more  than  one  visit  than  individuals  not  confident  in  the  provider. 
Table  10  shows  significant  interaction  effects  with  confidence  in  provider  for  education, 
diagnosis,  health  and  insurance.  The  effect  of  confidence  in  provider  on  the  probability 
of  a  subsequent  visit  was  between  9%  and  63%  weaker  for  individuals  with  a  high  school 
education  in  both  weighted  and  un-weighted  estimates  and  across  all  visits.  The  effect  of 
confidence  in  the  provider  was  300-500%  stronger  for  individuals  with  college.  The 
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importance  of  confidence  in  provider  was  60%  less  for  individuals  with  a  medical 
diagnosis  as  compared  to  individuals  with  a  mental  health  diagnosis.  The  effect  was  10% 
greater  for  individuals  reporting  excellent/very  good  health.  Interaction  terms  from  the 
weighted  model  indicate  the  importance  of  having  confidence  in  the  provider  is  greater 
for  individuals  with  private  insurance  and  between  38%  and  96%  less  important  for 
uninsured. 


Table  10  Significant  interaction  effects  with  confidence  in  provider  in  weighted  and  un-weighted 
models 


Main  effects 

Significant 
interaction  effects 

%  Net  effect  of  confidence  in  provider  on 
probability  of  subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

UNWEIGHTED 

Education 

-0.09* 

-0.55** 

-0.44* 

College 

4.60* 

3.29* 

Diagnosis  category 

Medical  diagnosis 

-0.59* 

WEIGHTED  a 

Education 

High  school 

•i — 

00 

m 

o 

i 

College 

5.20* 

4.21* 

Health  status 

Excel/very  good 

0.10* 

Insurance  coverage 

Private  ins 

1.67f 

Uninsured 

-0.38* 

-0.96** 

*  p<.05  **  p<.01  Approached  p<  05  a  variance  estimation  and  sampling  weights 


Appointment  difficulty/  Difficulty  contacting  provider  by  phone 

Appointment  difficulty  and  telephone  difficulty  were  not  significant  predictors  of  a 
subsequent  visit  in  either  weighted  or  un-weighted  models  and  I  question  whether  they 
belong  in  the  models  as  a  measure  of  satisfaction.  However,  interaction  models  do 
indicate  significant  differences  in  the  importance  of  these  two  variables  by  education, 
race  and  insurance  that  is  noteworthy.  Table  1 1  indicates  the  importance  of  not  having 
difficulty  obtaining  an  appointment  in  the  probability  of  a  subsequent  visit  was  60%  to 
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80%  less  important  for  individuals  with  higher  education.  For  individuals  with  private 
insurance  and  no  insurance  the  effect  of  appointment  difficulty  in  predicting  a 
subsequent  visit  is  negative,  but  for  individuals  with  public  insurance  the  effect  is 
significandy  stronger  in  predicting  a  subsequent  visit.  Separate  stratified  models  suggest 
publicly  insured  individuals  who  report  no  appointment  difficulty  are  almost  two  and  a 
half  times  more  likely  to  have  more  than  three  visits  than  individuals  who  report 
appointment  difficulty  (p<.05). 

Difficulty  contacting  the  provider  by  phone  was  significandy  more  important  for  the 
Hispanic  group  compared  to  white  and  African  American  groups.  The  net  effect  from 
interaction  models  suggest  the  effect  98%  to  152%  greater  for  Hispanic  individuals 
(Table  12).  Separate  models  stratified  by  race  found  Hispanics  who  report  no  phone 
difficulty  were  five  to  seven  times  more  likely  to  have  more  than  two  and  more  than 
three  visits  than  Hispanic  individuals  who  reported  phone  difficulty  (p<.05).  The  effect 
does  not  appear  to  be  the  result  of  small  sample.  These  findings  are  concerning  because 
they  suggest  barriers  to  obtaining  care  such  as  difficulty  getting  an  appointment  or 
difficulty  contacting  provider  by  phone  may  discourage  care  more  for  some  groups  such 
as  Hispanic  and  publicly/uninsured  groups.  At  the  very  least  the  findings  suggest 
language  problems  for  Hispanic  individuals  may  pose  greater  unique  barriers  to  care  via 
telephone  access  than  for  English  speaking  groups. 
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Table  11  Significant  interaction  effects  with  appointment  difficulty  in  weighted  and  un-weighted 
models 


Main  effects 

Significant 
interaction  effects 

%  Net  effect  of  difficulty  getting  an  appointment 
on  probability  of  subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

UNWEIGHTED 

Education 

Advanced  degree 

-0.74** 

Insurance 

Private  ins 

HHGSI9HI 

Public  ins 

WHMMM 

WEIGHTED 8 

Education 

Advanced  degree 

-0.84** 

-0.63t 

Insurance  coverage 

Public  insurance 

.82* 

1.35** 

Uninsured 

-.73* 

*  p<  05  **  p<  01 ^approached  p<.05  a  variance  estimation  and  sampling  weights 


Table  12  Significant  interaction  effects  with  telephone  difficulty  in  weighted  and  un-weighted 
models 


Significant 

Main  effects  interaction  effects 

%  Net  effect  of  telephone  difficulty  on  probability  of 
subsequent  visit 

MTlvis 

MT2vis 

MT3vis 

UNWEIGHTED 

0.98* 

Race  Hispanic 

WEIGHTED 8 

1.47* 

1.52* 

Race  Hispanic 

*  p<.05  *  *  p<  0 1  ^approached  p<.05  a  variance  estimation  and  sampling  weights 


Summary 

The  first  hypothesis  suggested  there  would  be  a  relationship  between  satisfaction  with 
care  and  the  probability  of  more  than  one,  more  than  two  and  more  than  three  visits. 
The  findings  suggest  there  is  a  relationship  between  satisfaction  and  the  probability  of  a 
subsequent  visit  in  several  models.  There  were  significant  main  effects  for  satisfaction 
with  quality  of  care  although  the  direction  was  surprising.  There  were  significant  main 
effect  for  satisfaction  with  provider/staff  in  models  both  with  and  without  analytic 
weights.  There  were  significant  main  effects  for  confidence  in  provider  primarily  in 
models  which  used  the  analytic  weights.  There  were  no  main  effects  for  provider  listens, 
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difficulty  getting  an  appointment,  or  difficulty  containing  provider  by  phone.  Significant 
findings  in  main  effects  models  were  primarily  in  predicting  more  than  one  visit  with 
litde  or  no  significance  thereafter  which  would  indicate  that  the  greatest  differences  in 
the  sample  are  between  those  with  only  one  visit  and  those  who  with  more  than  one 
visit.  There  were  a  greater  percentage  of  respondents  with  one  visit  who  had  less 
education,  better  mental  and  physical  health,  fewer  mental  health  diagnoses,  fewer 
activity  and  social  limitations,  less  psychotropic  medication  and  more  private  insurance, 
but  this  did  not  differ  dramatically  from  the  differences  seen  between  other  visits. 

The  interaction  models  detected  greater  between  group  differences  in  the  effects  of 
each  satisfaction  variable  on  the  probability  of  a  subsequent  visit  particularly  for  the 
probability  of  more  than  three  visits.  With  each  satisfaction  variable  there  was  a  strong 
pattern  of  interaction  effects  for  education,  health  and  diagnosis,  race,  age  and  insurance. 
In  this  analysis,  I  examined  the  relationship  between  satisfaction  and  the  probability  of  a 
subsequent  visit  focusing  on  the  first  four  visits  since  that  is  when  46%  of  the  sample  is 
lost.  The  next  task  was  to  model  the  relationship  between  satisfaction  and  level  of  use 
with  the  added  precision  of  a  targeted  population  of  respondents  with  depression  and  a 
compliance  measure  constructed  from  the  minimum  number  of  visits  and  number  of 
prescriptions  set  forth  in  depression  treatment  guidelines. 

A  split  sample  test  of  robustness  was  performed  for  the  main  effects  model 
predicting  more  than  one  visit  1)  to  evaluate  the  effect  of  retaining  the  left  censored 
cases  and  2)  as  assurance  that  respondents  beginning  care  in  Round  2  and  Round  3  were 
not  evaluating  satisfaction  differendy  since  their  evaluations  came  very  early  in  care. 


122 


There  were  no  differences  in  the  direction  or  magnitude  of  any  of  the  coefficients  using 
just  left  censored/Round  1  cases  compared  to  Round  2  and  Round  3  cases.  The  sample 
was  also  split  to  evaluate  the  influence  of  truncated  cases  on  the  relationship  between 
satisfaction  and  the  probability  of  a  subsequent  visit.  The  sample  was  too  small  to  run 
for  the  truncated  cases  alone  (N= 11 6)  so  I  compared  cases  not  truncated  with  the  entire 
sample  and  there  were  no  differences  in  the  strength  or  magnitude  of  the  coefficients. 
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Satisfaction  and  probability  of  depression  treatment  compliance 

A  weakness  in  the  previous  analytic  models  is  not  being  able  to  distinguish  individuals 
who  received  consultation  without  treatment  or  who  received  brief  treatment  from 
respondents  who  required  but  did  not  complete  treatment.  The  second  hypothesis 
explored  this  further  by  testing  the  assertion  that  there  would  be  a  relationship  between 
satisfaction  with  care  and  treatment  compliance  among  a  subgroup  of  respondents  with 
a  diagnosis  of  depression.  This  section  will  first  change  the  specifications  in  the  previous 
three  analytic  models  to  include  only  the  depressed  cohort.  This  will  illuminate  any 
differences  in  the  depressed  cohort  without  respect  for  whether  they  were  complaint. 
Second,  this  section  will  describe  differences  between  compliant  and  noncompliant 
depressed  respondents  in  terms  of  number  of  visits  and  antidepressant  prescriptions. 
The  covariates  used  earlier  will  be  re-evaluated  for  their  contribution  to  a  logistic 
regression  model  estimating  the  relationship  between  satisfaction  with  care  and  the 
probability  of  compliance  and  a  reduced  model  will  be  used  that  supports  the  smaller 
sample  size  of  depressed  cohorts. 

Without  respect  for  whether  they  were  compliant  the  cohort  of  depressed  patients 
was  examined  for  predictors  of  a  subsequent  visit.  The  model  was  significant 
p<.0001  (Table  13).  In  the  depression  cohort  females  and  individuals  with  college  were 
significantly  more  likely  to  have  more  than  one  visit.  Hispanic  and  African  Americans 
were  significant  less  likely  to  have  more  than  one  visit.  The  group  of  respondents  with 
depression  were  significantly  more  likely  to  report  poor  health.  Depressed  respondents 
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who  were  satisfied  with  the  quality  of  care  were  95%  less  likely  to  have  more  than  one 
visit  than  depressed  respondents  not  satisfied  with  the  quality  of  care.  Respondents  who 
were  satisfied  with  provider/ staff  were  three  and  a  half  times  more  likely  to  have  more 
than  one  visit.  The  depressed  group  was  different  from  the  total  sample  in  the  following 
ways:  Being  Hispanic  or  African  American  reduced  the  likelihood  of  a  subsequent  visit; 
being  in  poor  health  increased  the  likelihood  of  having  more  than  one  visit  six-fold  in 
the  depressed  group  compared  to  two  fold  in  the  total  sample.  Having  poor  mental 
health  did  not  increase  the  likelihood  of  having  more  than  one  visit  for  the  depression 
group  whereas  poor  mental  health  was  positive  and  significant  in  the  total  sample. 
Perhaps  this  was  because  the  entire  subgroup  had  poorer  mental  health.  Being  publicly 
insured  doubled  the  chances  of  having  more  than  one  visit.  Being  satisfied  with 
provider /staff  increased  the  probability  of  more  than  one  visit  almost  three  and  a  half 
times  for  the  depressed  cohort  compared  to  two  times  for  the  total  sample.  Satisfaction 
with  quality  of  care  reduced  the  probability  of  more  than  one  visit  for  both  groups. 


Table  13  Main  effects  model  of  the  relationship  between  satisfaction  with  care  and  the 
probability  of  more  than  one  visit  for  the  depression  subgroup 


:df=30) 

95%  Cl 

Sex 

Male 

Female 

2.94(.80)** 

1.07 

1. 7-5.0 

Race 

White 

Hispanic 

.43015)* 

-.84 

.22-.85 

Black 

.31015)* 

-1.18 

.12-.79 

Age 

25-44 

18-24 

.52024) 

-.65 

.21-1.3 

45-64 

.83024) 

-.18 

.48-1.46 

65-90 

.82029) 

-.20 

.41-1.67 

Education 

No  degree 

High  school/GED 

.98034) 

-.02 

.50-1.9 

Bachelor’s/other  degree 

2.70(1.2)* 

.99 

1.13-6.4 

Master’  s/Doctorate 

5.60(1.55) 

.95 

.80-8.4 

Insurance 

Private 

Public 

2.1(.75)* 

.74 

1.04-4.22 
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Uninsured 

.73(.41) 

-.32 

.24-2.22 

Health 

Excellent 

Very  good 

1.16(.49) 

.15 

.51-2.6 

Good 

1.55(.65) 

.44 

.68-3.53 

Fair 

4.8(2.4)** 

1.58 

1.84-12.7 

Poor 

6.26(3.9)** 

1.83 

1.8-21.3 

Mental  health 

Excellent 

Very  good 

1.45(.71) 

.37 

.55-3.8) 

Good 

1.3(.61) 

.26 

.52-3.24 

Fair 

.77(37) 

-.26 

.30-1.97 

Poor 

1.8(1.15) 

.59 

.52-6.3 

Activity  limits 

No 

Yes 

3.23(1.28)** 

1.17 

1. 5-7.0 

Social  limits 

No 

Yes 

.87(.41) 

-.14 

.35-2.2 

Psychotropic 

No 

1 

Yes 

1.41  (.40) 

.34 

.81-2.5 

Appt  difficulty 

Very  difficult 

No  difficulty 

1.15(.32) 

.14 

.67-1.98 

Phone  difficulty 

Very  difficult 

mm 

No  difficulty 

-.13 

.51-1.5 

Provider  listens 

No 

Yes 

2.28(1.57) 

.83 

.59-8.8 

Confident  in 

No 

Provider 

Yes 

1 .6(1.1) 

.47 

.41-6.2 

Satisfaction  with 

Not  satisfied 

provider/staff 

'■  Satisfied 

3.44(2.12)* 

1.24 

1.0-11.5 

Satisfaction  w 

Not  satisfied 

quality  of  care 

Satisfied 

.04(.04)** 

-3.24 

.004-.36 

Model  R2  =.33 


Probability  of  obtaining  a  X2  statistic  at  least  as  large  at  random:  *p<.05  **p<.01 


As  shown  above  to  compare  differences  between  the  depressed  cohort  and  the  total 
sample  I  simply  re-estimated  the  probability  of  more  than  one  visit  to  include  only  the 
depression  subgroup,  but  the  hypothesis  actually  calls  for  estimating  the  probability  of 
compliance  for  the  depressed  cohort.  The  compliance  outcome  measure  was  constructed 
such  that  respondents  with  a  diagnosis  of  depression  who  had  less  than  six  visits  or 
received  fewer  than  four  antidepressant  prescription  are  considered  noncompliant.  A 
drawback  is  that  using  minimal  treatment  standards  does  not  taken  into  account  partial 
compliance  such  as  receiving  three  of  four  prescriptions  or  having  five  instead  of  six 
visits.  T-tests  were  performed  comparing  the  mean  number  of  visits  and  prescriptions 
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between  compliant  and  noncompliant  patients  (Table  14).  Noncompliant  individuals 
had  fewer  visits  and  received  an  average  fewer  antidepressant  drug  prescriptions. 
Compliant  patients  had  a  wide  range  of  visits  so  the  distribution  was  examined  indicating 
50%  had  fewer  than  15  visits  and  75%  had  fewer  than  27  visits6. 


Table  14  Differences  in  visits  and  medication  between  compliant  and  non  compliant  respondents 


Compliant 
Mean  (SD) 

95%  Cl 

Non-compliai 
Mean  (SD) 

it 

95%  Cl 

t  statistic  (df) 

Visits 

Prescriptions 

23.5(26.9) 

12.7(7.5) 

21-26 

11-14 

2.36(1.4) 
2.2(1. 2) 

2.2-2. 5 
2.0-2.4 

-13.62  (753)* 
-14.08(282)* 

*  Significant  at  p<.05 


A  reduced  regression  model  was  fit  for  the  depression  subgroup  predicting  treatment 
compliance  because  of  the  reduction  in  sample  size.  As  before  a  process  of  variable 
selection  was  done  and  forward  selection  procedures  carried  out  to  learn  which  variables 
contributed  to  the  new  model.  The  group  of  sociodemographic  variables  did  not 
contribute  significandy  to  the  model.  Race  was  the  only  sociodemographic  variable  that 
was  significant.  The  group  of  impairment  variables  were  significant  at  p<.0001. 
Individually,  health,  psychotropic  drug  and  activity  limitations  were  significant  additions 
to  the  model  at  p<.05.  The  group  of  satisfaction  variables  contributed  significantly  to  the 
model  at  p<.05.  Individually,  only  the  contribution  of  appointment  difficulty  and  phone 
difficulty  to  the  model  approached  significance  at  p<.05.  The  full  model  made  no 
further  contribution  over  the  reduced  model  and  the  reduced  model  had  the  advantage 
of  not  reducing  power  lost  from  the  30%  reduction  in  sample  size  due  to  selection  of 
depression  subgroup  analysis.  The  reduced  model  containing  race,  age,  health,  activity 

6  T tests  and  comparisons  were  made  before  omitting  non-depression  ICD9  ‘296’  and  ‘300’ 
respondents 
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limitations,  satisfaction  variables  and  analytic  weights  is  shown  in  Table  15.  The  reduced 
model  was  significant  at  p<.0001.  Only  confidence  in  provider  was  significant  and  the 
direction  of  the  coefficient  was  negative  such  that  having  confidence  in  the  provider 
reduced  the  likelihood  of  compliance.  Also  note  the  direction  of  satisfaction  with  quality 
of  care  which,  although  not  significant,  was  now  positive.  This  was  seen  in  a  separate 
stratified  model  which  found  the  direction  of  satisfaction  with  quality  of  care  predicting 
compliance  was  positive  and  significant  for  individuals  having  a  high  school 
education/ GED  which  for  the  compliance  group  made  up  44%  of  the  sample. 


Table  15  Reduced  logistic  model  of  the  relationship  between  satisfaction  with  care  and  the 
probability  of  depression  treatment  compliance 


Ref  group  in  italics 

LR  X2  statistic  1 

(df=18 ) 

OR  (SE) 

Coefficient 

95%  Cl 

Race 

White/black 

Hispanic 

.42(.24) 

-.87 

.14-1.28 

Age 

25-44 

0-17 

,50(.31) 

-.69 

.15-1.7 

18-24 

1.01(.52) 

.01 

.37-2.8 

45-64 

1.02(.36) 

.02 

.51-2.03 

65-90 

.56(.25) 

-.58 

.23-1.35 

Health 

Excellent 

Very  good 

1.65(.65) 

.49 

.75-3.6 

Good 

2.4(1.28) 

.87 

.83-6.9 

Fair 

3.11(1.7)* 

1.14 

1. 0-9.2 

Poor 

4.8(2.61)* 

1.57 

1.6-14.0 

Activity  limitations 

No 

Yes 

1.67(.60) 

.51 

.82-3.4 

Appt  difficulty 

Very  difficult 

No  difficulty 

.27 

.65-2.6 

Phone  difficulty 

-.16 

.48-1.5 

Provider  listens 

No 

Yes 

1.73(1.1) 

.54 

.49-6.05 

Confident  in  provid 

No 

Yes 

.08(40)* 

-2.5 

.007-.90 

Satisfaction  with 

Not  satisfied 

provider/staff 

Satisfied 

15.2(.78) 

.42 

.55-4.2 

Satisfaction  w 

Not  satisfied 

quality  of  care 

Satisfied 

1.47 

.39-49. 

*p<.05 
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Interaction  terms  were  added  individually  into  the  main  effects  model.  Psychotropic 
drug  was  significant  in  both  weighted  and  un-weighted  models  for  compliance  (Table 
16).  The  effect  of  satisfaction  with  quality  of  care  and  having  confidence  in  the  provider 
was  weaker  for  respondents  on  a  psychotropic  drug.  The  finding  did  not  seem  to  result 
from  differences  in  the  depressed  cohort  which  were  shown  earlier  to  be  similar  to  the 
total  sample.  To  test  this  further,  the  reduced  model  was  run  for  the  probability  of  more 
than  five  visits  to  see  if  the  change  in  the  direction  of  the  relationship  was  related  to 
number  of  visits.  Satisfaction  with  quality  of  care  and  confidence  in  provider  were  not 
significant,  but  the  direction  of  the  coefficients  was  consistent  with  the  compliance 
model  with  satisfaction  with  quality  of  care  being  positively  related  to  compliance  while 
confidence  in  provider  being  negatively  related  to  compliance.  To  test  for  model 
misspecification  that  could  have  resulted  from  the  reduced  model,  the  reduced  model 
was  estimated  for  the  entire  sample  predicting  more  than  one  visit  to  see  if  the 
direction/ strength  of  the  coefficients  changed.  The  reduced  model  predicting  more  than 
one  visit,  even  without  controlling  for  gender,  education,  insurance  or  mental  health 
produced  estimates  that  were  identical  to  that  produced  by  the  full  model  predicting 
more  than  one  visit.  This  means  that  model  misspecification  from  the  reduced  model  is 
not  the  cause  for  differences  in  the  estimates  predicting  compliance. 
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Table  16  Significant  interaction  effects  for  compliance  in  un-weighted  models 


Main  effects 

Significant  interaction  effects 

%  Net  effect  on 
probability  of  treatment 
guideline  compliance 

COMPLIANCE 

UNWEIGHTED 

Psychotropic  Rx 

x  Satisfaction  with  quality  of  care 

.41* 

Psychotropic  Rx 

x  Confidence  in  provider 

.90* 

*  p<.05 


Summary 

It  seems  that  time,  e.g.,  having  more  than  five  visits,  more  than  the  condition,  e.g., 
depression,  or  the  effect  of  a  reduced  model  was  responsible  for  changes  in  the  direction 
of  the  relationship  between  satisfaction  variables  and  the  probability  of  compliance. 
There  appears  to  be  characteristics  unique  to  the  population  having  more  than  five  visits, 
aside  from  a  depression  diagnosis,  that  account  for  the  finding.  Aside  from  the  negative 
relationship  between  confidence  in  the  provider  and  probability  of  compliance,  there  was 
a  significant  interaction  between  confidence  in  provider,  satisfaction  with  quality  of  care 
and  psychotropic  drug.  The  effect  of  confidence  in  provider  an  satisfaction  with  quality 
of  care  were  weaker  for  individuals  on  a  psychotropic  drug.  This  was  the  finding  in  a 
separate  stratified  analysis  which  found  among  individuals  on  a  psychotropic  drug 
confidence  in  provider  had  a  significantly  reduced  effect  on  the  probability  of 
compliance  (Table  32).  This  implies  confidence  in  the  provider  matters  less  among 
individuals  on  a  psychotropic  drug.  Perhaps  this  is  because  individuals  place  their 
confidence  in  the  drug  treatment. 
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Satisfaction  with  care  and  time-to-failure  rates 

Thus  far  the  analysis  has  centered  around  satisfaction  with  care  and  the  probability  of 
continuation  in  the  first  four  visits  and  the  probability  of  treatment  guideline  compliance 
at  six  visits.  This  still  leaves  questions  about  whether  a  relationship  can  be  detected 
across  a  wider  interval  of  visits.  The  previous  model  predicting  depression  treatment 
compliance  hinted  that  the  relationship  between  satisfaction  and  the  probability  of  a 
subsequent  visit  may  not  be  constant  across  visits.  To  test  the  hypothesis  that  rates  of 
dropout  across  the  duration  of  care  are  influenced  by  satisfaction  with  care,  survival  or 
time  to  failure  methods  were  used  to  compare  failure  rates  of  satisfied  and  not  satisfied 
patients.  Survival  methods  are  used  in  research  to  determine  the  probability  and  timing 
of  an  event  such  as  dropout  to  occur.  The  survival  ‘function’  is  the  probability  that  the 
duration  until  an  event  occurs  is  of  at  least  length  t.  The  rate  at  which  the  duration  is 
completed  after  length  t,  given  that  they  lasted  at  least  until  t  is  called  the  hazard  rate  and 
indicates  the  failure  potential  (Kleinbaum,  1996).  The  higher  the  hazard  rate  the  lower 
the  probability  of  surviving.  For  this  analysis  time  to  dropout  is  the  outcome  variable  of 
interest  and  satisfaction  variables  are  the  predictors. 

Data  on  18,000  mental  health  visits  by  1,430  individuals  was  converting  to  survival 
time  data  using  the  stset  command  in  STATA.  The  distribution  of  failure  times  was 
estimated  across  the  sample  using  sts  graph.  The  functional  shape  of  the  hazard  function 
was  identified  as  an  increasing  Weibull  because  the  potential  for  dropout  increases  as  the 
number  of  visits  increase.  The  survivor  function  which  estimates  the  probability  of 
survival  over  time  indicates  75%  of  the  study  population  returned  after  the  first  visit,  less 
than  50%  returned  beyond  the  sixth  visit  and  20%  returned  beyond  20  visits. 
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Kaplan-Meier  estimator  curves  were  used  to  explore  differences  in  the  conditional 
failure  rates  predicted  by  each  satisfaction  variable.  There  were  only  slight  differences  in 
the  survival  function  between  high  and  low  satisfaction  for  satisfaction  with 
provider/staff,  confidence  in  provider,  appointment  difficulty  and  phone  difficulty. 
There  were  larger  differences  in  the  survivor  function  between  satisfaction  with  quality 
of  care  and  provider  listens.  For  example  the  failure  rate  when  provider  listens  was  73% 
as  compared  to  79%  when  the  provider  does  not  listen.  Satisfaction  with  quality  of  care 
shows  the  same  negative  relationship  with  continuation  as  seen  in  the  first  analytic 
models.  The  survival  rate  for  individuals  who  were  satisfaction  with  quality  of  care  was 
72%  as  compared  to  83%  for  individual  not  satisfaction  with  quality  of  care. 

The  Kaplan-Meier  curves  were  compared  for  statistical  differences  using  the  log-rank 
test  which  is  a  large  sample  chi-square  test  that  uses  expected  versus  observed  cell  counts 
over  categories  of  outcomes  to  evaluate  between  group  differences  in  survival  functions. 
The  survival  curves  did  not  differ  significantly  with  the  exception  of  provider  listens 
(p<.05).  Kaplan-Meier  survival  curves  crossed  for  each  of  the  variables  which  suggested 
that  control  variables  were  needed.  The  log  rank  test  was  again  used  to  evaluate  the 
effects  of  control  variables  on  each  of  the  six  satisfaction  variables.  There  were 
significant  differences  in  the  survival  functions  of  satisfaction  with  provider/staff  for 
males,  individuals  with  a  high  school  education,  private  insurance  coverage,  a  medical 
diagnosis  only  and  individuals  in  poor  health.  There  were  significant  differences  in  the 
survival  function  of  provider  listens  for  individuals  with  college,  for  the  age  group  45-64 
and  individuals  with  a  medical  diagnosis  only. 
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To  control  for  the  effects  of  gender,  race,  age,  education,  insurance,  diagnosis,  health, 
mental  health,  psychotropic  drug  and  truncation  (study  attrition),  adjusted  survival 
curves  were  used.  The  sample  was  also  capped  at  less  than  25  visits  to  reduce  the 
influence  of  extreme  outliers  (Edlund  et  al,  2002).  The  adjusted  curves  indicate 
differences  in  the  effect  of  satisfaction  on  survival  is  not  constant  with  respect  to 
number  of  visits.  For  example,  satisfaction  with  quality  of  care  appears  to  have  its 
greatest  impact  on  the  survival  rate  among  persons  with  between  five  and  15  visits 
(Figure  2).  Whereas  earlier  findings  suggested  that  high  satisfaction  with  quality  of  care 
was  associated  with  high  dropout  in  the  first  four  visits,  this  analysis  suggests  dropout  is 
lowest  for  individuals  who  are  highly  satisfied  with  quality  of  care  with  particularly  large 
differences  between  5  and  1 5  visits.  This  finding  was  conditional  on  controlling  for  age, 
health  and  insurance  coverage.  Without  these  three  control  variables  the  effect  on 
satisfaction  mimics  earlier  findings  where  high  satisfaction  with  quality  of  care  reduced 
the  probability  of  continuation  of  care7.  For  satisfaction  with  provider/staff  the 
differences  occur  between  two  and  eight  visits  but  with  a  wide  band  of  differences  across 
the  duration  of  care  (Figure  3).  Consistent  with  earlier  findings  high  satisfaction  with 
provider/ staff  has  a  higher  survival  rate.  Differences  in  provider  listens  has  its  greatest 
effect  for  persons  who  have  between  six  and  ten  visits  (Figure  5).  The  narrow  band 
suggests  few  differences  in  survival  rate  when  provider  listens.  This  is  consistent  with 

7  At  less  than  10  visits  high  satisfaction  with  quality  lowers  the  survival  rate  with  or  without 
controlling  for  age,  insurance  and  health.  At  less  than  15  visits  high  satisfaction  with  quality  increases 
the  survival  rate  with  or  without  age,  insurance  and  health.  At  less  than  20  visits  there  is  no  difference 
between  high  and  low  satisfaction  with  quality  without  age,  insurance  and  health.  With  the  control 
variables  high  satisfaction  results  in  lower  survival  rate.  At  less  than  26  visits  high  satisfaction  results 
in  higher  survival  rate  without  control  variables.  With  the  control  variables  high  satisfaction  results  in 
a  lower  survival  rate. 
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earlier  findings  of  few  significant  differences  in  the  first  four  visits  or  in  the  probability 
of  compliance. 

Because  the  survivor  function  for  this  data  follows  a  Weibull  distribution,  a  Weibull 
regression  model  was  favored  over  a  Cox  Proportional  hazards  model.  The  model  was 
fit  with  the  six  satisfaction  variables  and  covariates  were  added  to  the  model  to  capture 
adjusted  estimates  of  the  probability  of  discontinuation.  The  model  was  significant  at 
p<.0001  (table  not  shown).  In  survival  analysis  the  measure  of  effect  obtained  is  called 
the  hazard  ratio  which  is  the  time-to-failure  rate.  As  with  logistic  regression  it  is 
expressed  in  terms  of  an  exponential  of  a  regression  coefficient.  A  hazard  ratio  of  1 
means  there  is  no  effect.  A  hazard  ratio  of  10  means  that  an  exposed  group  (highly 
satisfied)  has  ten  times  the  hazard  or  failure  rate  of  the  unexposed  group  (not  satisfied). 
A  hazard  ratio  1/10  implies  the  exposed  group  (highly  satisfied)  has  one-tenth  the 
hazard  of  the  unexposed  group  (not  satisfied).  In  this  model  higher  education,  poor 
mental  health  and  being  on  a  psychotropic  drug  was  associated  with  a  decreased  risk  of 
dropout  at  p<.05  significance  level.  Truncation  (leaving  the  study  early)  (p<.01),  having 
no  diagnosis  (p<.01)  and  being  African  American  ^=.07)  were  associated  with  a  higher 
risk  of  dropout.  Having  confidence  in  provider  was  associated  with  a  decreased  risk  of 
dropout  but  this  not  quite  significant  (p=.08).  In  a  separate  model  not  controlling  for 
age,  health  and  insurance,  satisfaction  with  quality  of  care  and  confidence  in  provider 
both  reach  significance  at  (p<.05)  and  resulted  in  no  noticeable  changes  in  the  strength 
or  direction  of  the  hazard  ratios. 
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The  main  effects  survival  model  did  not  show  significant  effects  for  satisfaction. 
However,  adjusted  survival  curves  strongly  suggest  differences  exists  that  might  be 
explained  through  stratified  models.  A  summary  of  the  stratified  models  is  shown  in 
Table  34.  Consistent  with  the  previous  analysis  being  satisfied  with  the  quality  of  care 
decreased  the  chances  of  staying  in  treatment.  The  hazard  ratio  was  consistently  >  1 
across  most  groups.  Individuals  age  45-64  who  were  satisfied  with  quality  of  care  were 
nine  times  more  likely  to  leave  care  than  individuals  in  the  same  age  group  not  satisfied 
with  quality  of  care  (Figure  6).  Having  a  medical  diagnosis  only  (Figure  7)  or  advanced 
degree  were  also  associated  with  a  higher  hazard  ratio.  Individuals  with  an  advanced 
degree  had  50  times  the  hazard  rate  of  individuals  with  an  advanced  degree  not  satisfied 
with  quality  of  care  (Figure  8).  The  hazard  ratio  was  <1  for  the  Hispanic  group,  publicly 
insured  individuals  and  individuals  age  25-44  however  the  results  were  not  statistically 
significant.  Satisfaction  with  provider/ staff  was  associated  with  a  decreased  risk  of 
dropping  out  for  individuals  in  the  45-64  age  category  (Figure  9)  and  for  individuals  with 
an  advanced  degree  (Figure  10).  Being  satisfied  with  provider/staff  was  associated  with  a 
higher  risk  of  dropping  out  for  individuals  in  excellent/very  good  mental  health  (Figure 
11).  In  the  analytic  model  predicting  more  than  two  visits  this  group  were  statistically  less 
likely  to  drop  out. 

Unlike  the  previous  analysis  which  found  the  probability  of  a  subsequent  visit  only 
weakly  affected  by  the  variables  confidence  in  provider  and  provider  listens,  the  survival 
model  shows  more  pronounced  differences  for  these  variables  on  the  failure  rate.  Being 
in  good  health  and  reporting  confidence  in  the  provider  reduced  the  risk  of  dropout  by 
40%  compared  to  individuals  in  good  health  who  did  not  report  having  confidence  in 
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the  provider  (Figure  12).  For  individuals  with  good  mental  health  having  confidence  in 
provider  reduced  the  risk  of  failure  by  50%  (Figure  13).  Confidence  in  provider 
significandy  decreased  the  failure  rate  for  individuals  age  45-64  (Figure  14),  for 
individuals  with  an  advanced  degree  (Figure  15),  for  individuals  who  were  white  (Figure 
16)  and  for  individuals  with  no  diagnosis  (p=.07)  (Figure  17).  For  individuals  with  an 
advanced  degree,  provider  listens  also  significandy  decreased  the  failure  rate  (Figure  18). 
For  individuals  with  private  insurance  confidence  in  provider  reduced  the  failure  rate  by 
30%  ^=.07)  (Figure  19).  Provider  listens  was  associated  with  a  60%  reduction  in  the 
failure  rate  for  African  Americans  (Figure  20).  The  hazard  rate  was  significandy 
increasing  for  only  one  group:  Provider  listens  was  associated  with  an  increased  risk  of 
failure  for  individual  in  good  health  (p<.05)  (Figure  21). 

Summary 

In  summary  the  time  to  failure  rates  for  respondents  with  high  satisfaction  and  low 
satisfaction  were  not  statistically  different  in  the  main  effects  model,  but  there  were 
several  statistically  significant  differences  revealed  between  groups.  With  few  exceptions 
the  findings  were  consistent  with  logistic  regression  models  in  the  previous  analysis  with 
additional  findings  on  provider  listens  not  previously  found.  There  is  sufficient  support 
for  the  hypothesis  that  failure  rates  between  high  and  low  satisfaction  are  different  for 
African  Americans  and  whites,  individuals  in  excellent  health  and  mental  health,  age  45- 
64  and  individuals  with  an  advanced  degree.  As  seen  before  the  direction  of  the 
relationship  is  not  always  consistent.  Differences  in  the  finding  from  this  model  as 
compared  to  others  may  be  attributed  to  1)  the  use  of  slightly  different  set  of  covariates, 
2)  omitting  outliers  with  more  than  25  visits,  3)  differences  in  the  outcome  of  interest 
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failure  rate  over  the  duration  of  care  versus  probabilities  of  a  second,  third  or  fourth 
visit,  or  4)  the  use  of  stratified  models  without  a  priori  evidence  of  interactions. 

To  summarize  the  three  research  hypotheses  in  this  study  there  was  evidence  to 
suggest  a  relationship  does  exist  between  satisfaction  with  care  and  the  probability  of 
continuation.  There  were  some  main  effects  in  the  relationship.  However,  a  more 
interesting  story  that  emerged  was  the  extent  to  which  the  importance  of  satisfaction  in 
predicting  continuation  differed  across  the  sample  for  each  satisfaction  variable.  This 
was  evident  in  both  interaction  and  stratified  models.  The  stratified  models  indicated 
very  large  differences,  but  unless  there  was  a  corresponding  interaction  effect,  these  were 
de-emphasized  in  the  findings  because  of  the  greater  risk  of  making  of  Type  1  error. 
Even  with  the  more  conservative  approach  of  reporting  primarily  on  the  significant 
interaction  effects  there  were  still  significant  differences.  A  second  interesting  story 
comes  from  the  negative  direction  of  the  relationship  between  satisfaction  with  quality 
of  care  and  the  probability  of  continuation  beyond  one  visit.  Based  on  the  literature  and 
the  theory  it  was  expected  that  the  relationship  between  satisfaction  with  care  and  the 
probability  of  continuation  would  be  positive  for  all  six  satisfaction  variables,  but  this 
was  not  always  the  case.  Finally  to  comment  on  the  theoretical  framework.  Economic 
theory  served  as  an  adequate  model  for  evaluating  the  relationship,  but  would  have  been 
greatly  improved  by  testing  satisfaction  as  a  function  of  the  type  of  treatment  and  the 
probability  of  continuation  as  a  function  of  satisfaction.  The  final  chapter  provides 
further  discussion  of  the  findings  and  implications  for  research,  policy  and  practice. 
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CHAPTER  6  Discussion/Implications 
Major  findings 

The  goal  of  this  research  has  been  to  examine  the  relationship  between  satisfaction 
with  care  and  treatment  compliance.  Noncompliance  in  the  form  of  early  termination 
from  outpatient  mental  health  care  is  a  pervasive  problem  that  transcends  patient 
populations  and  clinical  settings.  Responsibility  for  the  problem  has  been  largely  placed 
on  the  patient.  Using  an  economic  framework  other  plausible  explanations  emerge  which 
suggest  noncompliance  is  the  patient’s  response  to  supply  side  factors.  The  supply  side 
of  mental  health  determines  not  only  the  costs  and  benefits  of  treatment,  but  the  extent 
to  which  patient  preferences  are  addressed  in  clinical  trials  and  incorporated  into  clinical 
practice  patterns.  Costs  and  benefits  have  been  well  studied,  but  the  notion  that  patient 
preferences  play  an  influential  role  in  decisions  about  continuation  of  care  has  received 
litde  attention.  By  examining  the  relationship  between  satisfaction  as  a  measure  of 
patient  preferences  and  continuation  with  care  as  a  measure  of  compliance  this  research 
has  sought  to  investigate  the  possibility  further.  In  this  section  I  will  provide  an  overview 
of  major  research  findings,  compare  the  findings  with  previous  research,  discuss 
strengths  and  weaknesses  of  the  research  design  and  performance  of  the  satisfaction 
measure.  Finally,  implications  for  social  work  policy,  research  and  practice  are  discussed. 

The  study  finds  strong  support  for  the  hypothesis  that  a  relationship  exists  between 
satisfaction  with  care  and  continuation  of  outpatient  mental  health  care.  After 
controlling  for  sociodemographic  characteristics  and  impairment,  satisfaction  had  a 
statistically  significant  influence  on  the  probability  of  more  than  one,  more  than  two  and 
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more  than  three  visits.  Satisfaction  had  a  significant  influence  on  time-to-failure  rates  and 
was  significantly  although  not  as  clearly  related  to  depression  treatment  compliance. 

Most  notably  satisfaction  with  care  does  not  appear  to  play  the  same  role  in 
predicting  treatment  continuation  for  everyone.  Having  confidence  in  the  provider  and 
being  satisfied  with  the  provider /staff  increased  the  probability  of  a  subsequent  visit  for 
those  discontinuing  care  in  the  first  four  visits.  Being  satisfied  with  the  quality  of  care 
reduced  the  probability  of  a  subsequent  visit  for  those  discontinuing  care  in  the  first  four 
visits  with  the  sharpest  differences  between  individuals  having  one  visit  and  individuals 
having  more  than  one  visit.  This  suggests  that  for  many  individuals  an  important 
outcome  is  attained  at  the  first  visit  for  which  individuals  are  satisfied  reducing  the 
probability  of  a  subsequent  visit.  Having  confidence  in  the  provider  reduced  the 
probability  of  complying  with  the  depression  treatment  guidelines  for  individuals  on  a 
psychotropic  drug,  but  satisfaction  with  quality  of  care  increased  the  probability  of 
compliance  for  individuals  with  a  high  school/ GED  education.  Kaplan  Meier  survivor 
curves  indicate  the  greatest  band  of  differences  in  the  continuation  rate  of  individuals 
who  were  satisfied  and  individuals  not  satisfied  with  care  occurred  as  late  as  between  10 
and  15  visits  suggesting  the  relationship  between  satisfaction  and  the  probability  of 
continuation  may  depend  on  the  number  of  visits  already  attained. 

Although  the  sociodemographic  profiles  of  individuals  who  discontinued  care  early 
did  not  differ  dramatically  from  those  who  continued  in  care,  the  degree  to  which 
satisfaction  affected  the  probability  of  a  subsequent  visit  varied  dramatically  by 
education,  age,  health  status,  mental  health  diagnosis,  age,  insurance  and  race. 
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Satisfaction  was  particularly  an  important  factor  for  the  mental  health  population  where 
individuals  satisfied  with  provider/ staff  were  three  and  a  half  times  more  likely  to  have 
more  than  one  visit  compared  to  twice  as  likely  for  the  entire  sample.  Being  satisfied 
with  the  quality  of  care  and  having  confidence  in  the  provider  was  significantly  and 
consistently  less  important  for  individuals  with  a  high  school  education  and  significantly 
more  important  for  individuals  with  college.  For  African  Americans,  individuals  age  25- 
44  and  for  individuals  with  a  high  school  education  the  effect  of  satisfaction  with  quality 
was  significantly  less  important  in  predicting  continuation.  Contrary  to  my  early 
questions  of  the  validity  of  telephone  difficulty  and  appointment  difficulty  questions, 
there  were  some  interesting  findings.  Not  having  difficulty  contacting  the  provider  by 
phone  significantly  increased  the  likelihood  of  having  more  than  two  and  more  than 
three  visits  for  the  Hispanic  group.  Similarly,  not  having  difficulty  getting  an 
appointment  increased  the  likelihood  of  having  more  than  two  and  more  than  three 
visits  for  individuals  who  were  publicly  insured,  but  was  significantly  less  important  for 
individuals  with  a  Master’s  degree  or  greater,  privately  insured  and  uninsured  individuals. 

The  findings  in  this  study  with  regard  to  mental  health  utilization  rates  were 
consistent  with  previous  research.  For  example,  Olfson  &  Pincus  (1994a)  found  15%  of 
individuals  with  at  least  one  visit  had  more  than  20  visits.  The  current  study  found  19% 
of  individuals  with  one  visit  had  more  than  20  visits.  Kendall  &  Sugarman  (1997)  report 
23%  of  patients  dropout  after  the  first  visit  and  70%  dropout  by  the  tenth  session.  The 
current  study  found  23%  dropout  after  the  first  visit  and  66%  dropout  by  the  tenth  visit. 
Low  mental  health  utilization  among  African  Americans  has  been  demonstrated  in  past 
studies  (Hu  et  al.,  1991;  Wierzbicki  &  Pekarik,  1993;  Grunebaum  et  al.,  1996;  Swartz  et 
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al.,  1998).  These  findings  were  supported  in  the  current  data  where  African  Americans 
had  the  fewest  number  of  visits  on  average  and  had  a  very  low  probability  of  a 
subsequent  visit  in  almost  every  analytic  model. 

Previous  studies  have  found  females  to  be  heavy  utilizers  of  outpatient  mental  health 
care  (Horgan,  1983;  K.  Wells  et  al.,  1986;  Olfson  &  Pincus,  1994b;  Frieman  &  Zuvekas, 
2000).  In  this  study  females  were  more  likely  to  have  a  subsequent  mental  health  visit  but 
males  and  females  on  average  had  the  same  number  of  visits.  The  percentage  of  males  in 
the  study  population  increased  from  38%  having  more  than  one  visit  to  41%  having 
more  than  15  visits.  Only  males  with  a  high  school  education  or  less,  in  poor 
physical/mental  health,  and  males  who  were  publicly  insured/uninsured  had  a  greater 
likelihood  of  a  subsequent  visit  than  females.  Wang  et  al  (2000)  found  males  more  likely 
than  females  to  be  compliant,  but  this  finding  was  not  supported  in  the  depression 
treatment  compliance  group. 

R.  C.  Kessler  (1994)  found  the  prevalence  of  mental  disorder  to  be  highest  among 
individuals  between  25  and  44  years  old  and  declining  thereafter.  The  current  data  found 
this  age  group  had  one  of  the  highest  average  number  of  visits-  second  only  to  the  0-17 
age  group.  However,  satisfaction  with  care  was  not  important  in  predicting  a  subsequent 
visit  for  25-44  year  olds  and  the  effect  of  satisfaction  was  generally  weaker  for  younger 
age  groups.  Being  older  has  been  previously  associated  with  greater  satisfaction  (Hoff  et 
al.,  1999;  Fontana  &  Rosenheck,  2001).  Satisfaction  was  a  very  important  predictor  for 
the  45-64  age  group  who  had  almost  as  many  visits  on  average  as  25-44  year  olds  but  for 
whom  satisfaction  with  care  was  a  very  significant  predictor.  The  elderly  have  been 
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previously  shown  to  be  particularly  reluctant  to  use  mental  health  care.  This  is 
consistent  with  the  oldest  age  group  in  the  sample  where  individuals  65-90  had  the 
lowest  average  number  of  visits  even  among  the  depression  treatment  compliance  sub¬ 
group.  Nonetheless  in  the  first  15  visits,  the  proportion  of  65-90  year  olds  still  increased 
from  13.8%  having  more  than  one  visit  to  17%  having  more  than  15  visits. 

Higher  education  has  been  associated  with  increased  mental  health  use  (Frieman  & 
Zuvekas,  2000;  Mueller  &  Pekarik,  2000)  and  greater  compliance  (Wierzbicki  &  Pekarik, 
1993;  Centorrino  et  al.,  2001).  In  the  current  sample  individuals  with  a  bachelor’s 
or  advanced  degree  had  the  greatest  number  of  visits  than  almost  any  other  group  with 
16  and  19  visits  respectively.  In  the  depression  treatment  compliance  subgroup 
individuals  with  an  advanced  degree  had  on  average  25.9  visits-  far  more  than  individuals 
reporting  even  the  most  severe  impairment  at  19  visits.  Although  those  with  a  higher 
education  had  more  visits,  individuals  with  a  high  school/GED  education  represented 
the  largest  proportion  in  the  sample  and  made  up  the  greatest  percentage  of  individuals 
who  were  compliant  at  44%.  Among  individuals  with  college  having  confidence  in  the 
provider  was  a  particularly  significant  predictor  of  a  subsequent  visit  but  had  a  negative 
influence  on  the  probability  of  more  than  one  visit  for  individuals  with  a  high  school 
education.  Similarly,  satisfaction  with  quality  of  care  was  an  important  predictor  for 
individuals  with  college,  but  was  a  significantly  less  important  predictor  for  individuals 
with  high  school.  Satisfaction  with  provider/ staff  was  significantly  more  important  in 
predicting  more  than  one  visit  for  individuals  with  an  advanced  degree  compared  to  all 


other  levels  of  education. 
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Mental  health  use  has  been  associated  with  insurance  coverage  with  lack  of  coverage 
thought  to  impede  access  to  mental  health  care.  In  the  current  data  individuals  with 
public  insurance  had  the  greatest  average  number  of  visits  while  the  privately  insured 
individuals  had  the  fewest  number  of  visits.  Satisfaction  with  care  was  predictor  of  a 
subsequent  visit  for  privately  insured  and  uninsured  in  stratified  and  interaction  models. 
For  publicly  insured  individuals  confidence  in  provider  was  significantly  less  important  in 
predicting  a  subsequent  visit  while  having  no  appointment  difficulty  had  a  significandy 
stronger  effect  on  the  probability  of  a  subsequent  visit. 

The  literature  suggests  poor  health  is  associated  with  a  greater  probability  of  any 
mental  health  use,  but  better  health  is  associated  with  level  of  use.  In  the  current  sample 
individuals  with  poorer  health,  activity  limitations,  and  a  medical  diagnosis  had  a  greater 
average  number  of  visits  than  individuals  reporting  better  health,  no  activity  limitations, 
and  no  diagnosis.  This  was  not  the  case  in  the  depression  treatment  compliance 
subgroup  which  found  individuals  reporting  excellent  health  had  as  many  visits  as 
individuals  reporting  fair  or  poor  health.  Individuals  in  excellent  health  made  up  65%  of 
the  sample  having  more  than  one  visit  but  only  57%  having  more  than  15  visits. 
Satisfaction  with  care  was  not  a  predictor  of  a  subsequent  visit  for  individuals  in  poor 
heath,  but  the  effect  of  satisfaction  was  significantly  weaker  in  predicting  a  subsequent 
visit  for  individuals  in  excellent  health. 

Mental  status  has  been  shown  to  be  the  strongest  predictor  of  mental  health  use 
(Flaskerud  &  Hu,  1992;  Landerman  et  al.,  1994;  R.  C.  Kessler  et  al.,  1998).  In  the  current 
data  individuals  reporting  poor  mental  health  and  individuals  with  a  mental  health 
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diagnosis  had  the  greatest  number  of  visits.  Individuals  reporting  poor  mental  health 
represented  40%  of  the  population  having  more  than  one  visit  and  increasing  to  49% 
having  more  than  15  visits.  Satisfaction  with  care  was  not  a  significant  predictor  of  a 
subsequent  visit  for  individuals  in  poor  mental  health,  but  it  was  a  strong  positive 
predictor  for  individuals  with  a  mental  health  diagnosis.  Satisfaction  with  quality  of  care 
had  four  times  the  effect  on  the  probability  of  a  subsequent  visit  for  individuals  with  a 
mental  health  diagnosis  whereas  not  having  a  mental  health  diagnosis  significantly 
weakened  the  effect  of  satisfaction  with  quality  of  care  on  the  probability  of  a 
subsequent  visit. 

Study  strength  and  weaknesses 

There  are  several  significant  limitations  to  this  study.  This  section  will  summarize 
weaknesses  and  strengths  of  the  design  and  measures.  However,  there  are  several  general 
criticisms  that  can  be  made  outright.  The  first  concern  called  into  question  with  any 
satisfaction  study  is  that  many  respondents  may  be  naturally  inclined  toward  pessimism 
or  optimism  irrespective  of  their  care  experience.  Evaluations  of  satisfaction  with  care 
would  be  terminally  distorted  by  these  overly  positive  or  overly  negative  attitudes.  This  is 
thought  to  be  especially  problematic  for  the  mental  health  population  where  many 
mental  disorders  are  characterized  by  sadness,  discontent  and  at  the  extreme  perceptual 
and  emotional  disturbance.  As  was  addressed  in  the  literature  there  are  researchers  who 
question  the  validity  of  satisfaction  measurement  at  all  for  mental  health  population.  I  do 
not  subscribe  to  this  view  and  find  no  support  for  in  the  data  to  indicate  the  measure 
was  not  valid.  A  second  concern  is  the  omitted  variable  problem  that  arises  in  all  social 
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science  research.  The  argument  goes  that  because  social  science  researchers  operate 
outside  a  laboratory  environment  and  on  human  conditions,  we  can  never  be  completely 
sure  there  is  not  some  important  variable  operating  that  was  not  measured  in  the  study. 
Scholars  argue  that  because  of  the  ever-present  possibility  of  an  omitted  variable,  an 
'effect5  such  as  suggested  in  this  study  can  never  really  be  demonstrated. 

A  major  difference  between  this  study  and  research  previously  carried  out  on 
satisfaction  with  care  is  the  study  design.  This  section  discusses  strengths  and  weaknesses 
of  the  study  design,  choice  of  outcome  variables  and  performance  of  the  satisfaction 
measure  as  a  predictor  of  continuation.  The  secondary  analysis  of  data  from  national 
probability  studies  is  commonly  used  for  mental  health  utilization  research  carried  out  in 
the  U.S.,  but  has  not  been  used  for  research  on  satisfaction  with  care  in  health  or  mental 
health  care.  Employing  a  national  sample  for  this  study  had  two  advantages.  First,  it 
yielded  a  larger  sample  than  has  been  attained  in  previous  satisfaction  with  care  studies 
enabling  a  more  detailed  analysis.  Second,  it  produced  findings  that  are  generalizeable  to 
the  U.S.  population  increasing  the  relevance  across  patient  populations.  There  were  six 
disadvantages.  First,  the  extent  to  which  respondent  satisfaction  evaluations  of  their 
mental  health  care  were  successfully  isolated  from  their  evaluations  of  health  care 
experiences  was  unknown. 

Second  the  patient  satisfaction  measure,  while  an  industry  standard,  is  not 
theoretically  correct  for  evaluating  patient  preferences  for  treatment.  Third,  because  I 
had  no  information  to  differentiate  between  treatments  except  those  involving 
psychotropic  medication  the  study  could  not  focus  on  whether  treatment  type  affected 
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either  satisfaction  with  continuation  rates.  I  could  evaluate  only  whether  satisfaction 
influenced  continuation.  Fourth,  although  MEPS  provides  comprehensive  data  on 
mental  health  utilization  continuation  of  care  rates  are  still  an  imperfect  proxy  for 
treatment  compliance.  Fifth,  each  visit  in  the  study  is  presumed  to  be  part  of  a  single 
continuous  episode  of  care.  This  is  not  likely  given  tight  limits  on  number  of  visits 
imposed  by  most  third  party  payers.  Finally,  there  is  the  possibility  that  multiple 
providers  were  delivering  care  even  within  a  single  episode.  For  example,  in  mental 
health  care  a  psychiatrist  or  family  physician  may  prescribe  the  medication  while  a 
psychologist  or  social  worker  provides  the  psychological  treatment. 


Outcome 

In  this  study  the  effect  of  satisfaction  on  continuation  of  care  was  evaluated  against 
three  different  sets  of  outcomes:  the  probability  of  more  than  two,  more  than  three  and 
more  than  four  visits,  the  probability  of  depression  treatment  compliance,  and 
differences  in  the  hazard  rate  between  high  and  low  satisfaction.  This  offered  a  snapshot 
of  the  effects  of  satisfaction  in  predicting  continuation  for  consumers  at  different  levels 
of  use  and  findings  suggests  the  relationship  changes  over  level  of  use.  When  the  level  of 
use  is  low  satisfaction  with  quality  of  care  is  a  significant  negative  predictor  and 
confidence  in  the  provider  is  a  significant  positive  predictor.  For  individuals  consuming 
more  than  five  visits  having  confidence  in  the  provider  reduces  the  probability  of 
continuation  while  satisfaction  with  quality  becomes  a  positive  predictor  with  high 
satisfaction  with  quality  having  its  greatest  impact  on  continuation  for  individuals 
consuming  between  5  and  15  visits. 
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This  suggests  there  are  at  least  two  distinct  types  of  consumers  of  outpatient  mental 
health  services:  those  who  discontinue  care  after  one  or  two  visits  and  those  who 
continue.  The  effect  of  satisfaction  is  strongest  predicting  more  than  one  visit  and 
becomes  weaker  thereafter.  This  along  with  the  finding  that  satisfaction  with  quality  of 
care  reduces  the  probability  of  more  than  one  visit  suggests  that  information  and 
reassurance  obtained  from  consultation  alone  may  itself  be  a  desirable  end  product  for 
some  individuals.  At  one  visit  the  probability  of  returning  is  greater  for  individuals  with 
confidence  in  the  provider  and  satisfaction  with  provider/staff,  but  the  probability  is 
reduced  for  individuals  satisfied  with  the  quality  of  care.  In  contrast  are  individuals  who 
continue  care  beyond  the  initial  visits.  Among  individuals  continuing  care  beyond  five 
visits  confidence  in  the  provider  had  a  significantly  negative  effect  particularly  among 
individuals  on  a  psychotropic  drug.  Additionally  the  effect  of  satisfaction  with  the  quality 
of  care  no  longer  reduced  the  probability  of  a  subsequent  visit  beyond  five  visits,  and  for 
individuals  with  a  high  school  education  it  significantly  increases  the  probability  of  a 
subsequent  visit. 

Satisfaction  measure 

The  satisfaction  with  care  measure  found  patient  satisfaction  to  be  consistently  and 
strongly  important  for  individuals  with  the  greatest  need—  namely  those  with  a  mental 
health  diagnosis.  Satisfaction  with  care  had  a  significantly  weaker  effect  for  those  with 
the  less  need — individuals  without  a  mental  health  diagnosis  and  individuals  reporting 
better  health  and  mental  health.  The  consistency  in  the  findings  provide  good  support 
for  the  validity  of  the  measure  with  the  mental  health  population.  There  are  several  other 
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lessons  with  regard  to  the  satisfaction  measure  that  are  discussed  in  this  section.  First, 
satisfaction  with  care  had  little  effect  beyond  the  probability  of  more  than  one  visit  in  the 
main  effects  models.  This  is  consistent  with  research  findings  from  consumer  and 
marketing  literature  which  finds  customer  satisfaction  measures  do  not  perform  well  for 
pooled  samples.  This  is  because  preferences,  which  often  follow  sociodemographic 
patterns,  wash  out  in  the  aggregate.  Interestingly  this  point  never  came  up  in  the  mental 
health  satisfaction  studies  reviewed  where  design  issues  of  global  versus 
multidimensional  measures  were  the  focal  point. 

Second  the  satisfaction  measure  revealed  important  differences  between 
sociodemographic  groups  in  predicting  a  subsequent  visit.  The  measure  worked  well  for 
higher  education,  privately  insured,  but  had  no  effect  or  had  opposite  effect  for  African 
American,  individuals  with  less  education  and  publicly  insured.  This  suggests  one  of  two 
possibilities.  Either  an  outside  factor  was  affecting  continuation  rates  for  some  groups 
that  were  not  controlled  for  in  the  analytic  models  such  as  income,  family  support  or 
treatment  type,  or  perhaps  the  satisfaction  measure  simply  did  not  tap  into  the  important 
questions  for  some  groups.  For  example,  ‘quality  of  care’  may  be  a  culturally  derived 
construct  that  does  not  measure  well  across  income,  education  or  race/ethnicity.  The 
satisfaction  variables  which  represented  more  practical  aspects  of  care  (telephone  and 
appointment  difficulty)  were  significantly  more  important  predictors  for  these  groups. 
This  begs  the  question  of  whether  the  satisfaction  with  care  is  important  at  all  to  these 
groups,  or  whether  it  is  important  but  inadequately  represented  by  the  measure. 
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Third  the  study  overcomes  three  common  measurement  problems:  Selection  bias, 
acquiescence  and  discriminant  validity.  Selection  bias  is  a  common  problem  where 
sample  losses  due  to  dropout  create  a  bias  toward  completers.  This  problem  was 
overcome  by  using  a  household  based  survey  which  provided  access  to  mental  health 
users  outside  the  treatment  episode  or  setting.  However,  the  choice  of  using  a 
household  sample  where  the  satisfaction  measure  administered  independent  of  a 
treatment  episode  came  at  the  cost  of  specificity  as  individuals  may  have  based 
satisfaction  evaluations  on  non-mental  health  care  experiences.  The  visit  weights  were 
constructed  to  minimize  the  influence  of  individuals  who  overall  had  a  greater  ratio  of 
health  care,  but  this  was  nonetheless  a  study  weakness8.  Acquiescence  is  another 
problem  seen  in  previous  research  on  satisfaction  where  responses  are  universally 
positively  skewed.  In  this  study  responses  were  skewed  toward  high  satisfaction  but  with 
the  large  sample  and  many  control  variables  the  measure  still  found  differences  in 
outcome.  Finally  in  terms  of  validity  the  six  questions  that  comprised  the  satisfaction 
measure  appeared  to  discriminate  between  different  aspects  of  satisfaction.  Even 
questions  that  seemed  conceptually  closely  related  such  as  appointment  and  phone 
difficulty,  and  confidence  in  provider,  satisfaction  with  provider/staff  and  provider 
listens  measured  as  unique  constructs.  This  provides  support  for  satisfaction  as  a 
multidimensional  measure,  but  still  raises  questions  about  whether  the  constructs  used  in 
this  study  are  ones  which  are  most  important  to  patients  in  predicting  future  care. 

Economic  framework 


8  This  was  examine  further  by  re-estimating  regression  models  to  include  those  with  only  mental  health 
visits  finding  the  direction  and  strength  of  the  coefficients  to  be  unchanged. 
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The  economic  framework  used  in  the  study  provided  a  compelling  new  way  of 
understanding  a  very  old  problem.  However  the  study  design  did  not  do  justice  to  the 
model.  First,  the  theoretical  model  calls  for  evaluating  patient  preferences  and  for  this 
patient  satisfaction  is  not  a  very  useful  proxy.  Satisfaction  measures  do  not  typically 
account  for  opportunity  costs — a  key  ingredient  in  revealed  preference  models.  This 
means  it  is  not  enough  to  know  if  the  patient  is  satisfied  without  knowing  whether  that 
aspect  of  care  for  which  they  were  satisfied  is  important  to  them  to  begin  with.  Second, 
the  data  did  not  support  enough  information  to  evaluate  supply  side  factors — such  as 
type  of  provider  and  type  of  therapeutic  intervention  (other  than  medication).  Third, 
additional  data  was  needed  in  order  to  distinguish  between  separate  episodes  of  care. 
This  study  employed  rough  measures  only  and  could  not  eliminate  left  censored  cases 
because  of  sample  size  reduction.  Finally,  the  economic  framework  was  not  being 
empirically  tested  against  any  other  model.  Therefore  I  cannot  make  any  comparative 
statement  about  its  performance  relative  to  other  frameworks.  The  hypotheses  might 
have  been  refined  to  test  differences  in  effect  of  satisfaction  within  each  subgroup  since 
the  interaction  effects  found  were  robust  and  might  have  included  an  income  covariate. 


Implications  for  research  and  policy  and  practice 

There  are  broad  implications  from  these  findings  that  are  relevant  to  practice, 
research  and  policy.  The  first  is  that  consumers  of  mental  health  care  are  not  a 
homogenous  group  in  terms  of  decisionmaking  about  continuation  of  care.  The  research 
finds  evidence  of  a  distinct  group  of  mental  health  consumers  who  terminate  after  only 
one  visit  apparently  satisfied  with  the  outcome.  For  others  satisfaction  with  care  is 
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important,  but  the  role  of  satisfaction  in  predicting  continuation  still  varies  by 
sociodemographic  factors,  degree  of  impairment  and  level  of  use.  The  degree  to  which 
we  understand  the  relationship  between  satisfaction  with  care  and  continuation  with  care 
is  sensitive  to  conceptualization  and  measurement  of  satisfaction.  This  research  suggests 
that  for  some  groups  practical  aspects  of  care  such  as  appointment  and  telephone  access 
are  more  important  in  decisions  about  continuation  than  having  confidence  in  the 
provider  or  being  satisfied  with  provider/ staff.  Measuring  patient  satisfaction  with  care  is 
important,  but  drawing  out  the  divergent  valuations  for  treatment  and  the  treatment 
environment  is  even  more  important.  Development  of  preference  measures  that  account 
not  only  for  whether  the  patient  was  satisfied  with  an  aspect  of  care,  but  whether  that 
aspect  of  care  was  important  to  begin  with  should  be  a  central  goal. 

There  are  three  recommendations  for  future  research.  First,  the  extent  to  which 
patients  follow  through  with  psychological  treatments  needs  to  be  a  central  outcome 
measure  in  research  on  the  efficacy,  effectiveness  and  cost  benefit  of  treatments.  Second 
methodologies  need  to  be  developed  to  support  measurement  of  treatment  preferences 
and  these  methodologies  utilized  in  research  from  which  new  treatments  are  developing. 
Third  and  even  more  fundamentally  the  use  of  economic  theory  to  model  the 
relationship  between  treatment  method,  satisfaction  and  compliance  warrants  further 
testing.  Even  in  this  study  where  the  fidelity  of  the  model  fell  short  because  of  imperfect 
design  and  measures,  the  findings  nonetheless  suggest  that  decisions  about  continuation 
could,  along  with  other  factors,  be  a  function  of  patient  valuations  for  care.  Further 
research  might  develop  and  test  an  econometric  model  of  the  demand  for  level  of 
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outpatient  mental  health  care  using  some  other  measure  of  patient  valuations  to  predict 
demand. 

Although  there  are  boundless  opportunities  to  use  research  to  advance  out 
knowledge  of  noncompliance  and  preferences  for  treatment,  there  are  also  important 
implications  for  practice.  In  the  practice  environment  it  is  important  to  recognize  that 
the  application  of  global  patient  satisfaction  measures  serves  little  purpose  because  these 
do  not  reveal  dissatisfaction  with  care.  Satisfaction  measures  require  redesign  to  yield 
useful  feedback  in  the  absence  of  large  samples  and  that  address  the  aspects  of  care  that 
are  important  to  less  adequately  served  populations.  Addressing  preferences  for 
treatments  directly  with  patients  in  practice  settings  could  reduce  early  termination  for 
some,  but  given  that  patient  preferences  were  not  a  key  component  in  the  development 
of  existing  treatments,  addressing  preferences  among  treatments  patients  might  not  have 
preferred  to  begin  with  may  not  yield  any  significant  reduction  in  early  terminations. 
Finally,  from  an  administration  perspective,  elaborate  mechanisms  and  standards  for  the 
labor  intensive  assessment  and  treatment  planning  of  every  patient  when  40%  may  only 
be  seeking  information  and  reassurance  may  not  be  a  practice  that  adds  to  the  quality  of 
outpatient  mental  health  care.  The  costs  and  benefits  of  this  practice  need  to  be  re¬ 
examined. 

There  are  policy  implications.  The  first  is  that  the  industry  needs  to  capitalize  on  the 
informational  value  of  economic  behavior  such  as  treatment  noncompliance  and  other 
utilization  patterns  in  the  design  of  mental  health  care  delivery  systems.  Noncompliance 
sends  a  strong  message  about  the  appeal  of  the  mental  health  system  beyond  the  initial 


152 


visits  where  the  marginal  value  of  care  decreases  rapidly.  Better  knowledge  about  what 
patients  value  most  relative  to  what  they  value  least  could  aid  in  the  trade-off  of 
resources  that  make  up  the  mental  health  service  delivery  system.  Concerns  about 
underutilization  of  mental  health  care  among  underserved  populations  could  also  be 
addressed  through  more  information  about  preferences  of  these  individuals.  This  study 
had  many  weaknesses  and  serves  only  as  a  point  of  departure  for  more  rigorous  research 
into  the  role  of  patient  preferences  in  treatment  compliance  beyond  this  study  and  to 
further  investigate  the  usefulness  of  economic  models  in  mental  health. 
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Appendix  A 


Table  17  Descriptive  statistics  for  satisfaction  with  care 


Descriptives  (N=1433) 

Freq 

Percentage 

Appointment  difficulty 

1  Not  at  all  difficult 

155 

10.82% 

2  Not  too  difficult 

258 

18.00% 

3  Somewhat  difficult 

459 

32.03% 

4  Very  difficult 

531 

37.06% 

Missing 

30 

2.09% 

Telephone  difficulty 

4  Not  at  all  difficult 

156 

10.91% 

3  Not  too  difficult 

223 

15.59% 

2  Somewhat  difficult 

474 

33.15% 

1  Very  difficult 

488 

34.13% 

Missing 

89 

6.22% 

Provider  listens 

1  Yes 

1359 

95.03% 

2  No 

55 

1.12% 

Missing 

16 

3.85% 

Confidence  in  Provider 

1  Yes 

1338 

93.57% 

2  No 

78 

5.45% 

Missing 

14 

.98% 

Satisfied  with  provider/staff 

1  Very  satisfied 

1000 

69.93% 

2  Somewhat  satisfied 

341 

23.85% 

3  Not  too  satisfied 

55 

3.85% 

4  Not  at  all  satisfied 

23 

1.61% 

Missing 

11 

.77% 

Satisfied  with  quality  of  care 

1  Very  satisfied 

1079 

75.45% 

2  Somewhat  satisfied 

291 

20.35% 

3  Not  too  satisfied 

39 

2.73% 

4  Not  at  all  satisfied 

13 

.91% 

Missing 

8 

.56% 
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Table  18  Descriptive  statistics  of  sociodemographic  characteristics 


Descriptives  (N=1433) 

Freq 

P 

Age 

1  0-4 

14 

0.98% 

2  5-17 

225 

15.70% 

3  18-24 

92 

6.42% 

4  25-44 

545 

38.03% 

5  45-64 

406 

28.33% 

6  65-90 

151 

10.54% 

Gender 

1  Male 

546 

38.10% 

2  Female 

887 

61.90% 

Race 

1  Hispanic 

208 

14.52% 

2  Black/Not  Hispanic 

126 

8.79% 

3  Other 

1099 

76.69% 

Education 

1  No  degree 

225 

17.79% 

2  High  school  diploma/ GED 

600 

41.87% 

3  Bachelor’s/Other  degree 

271 

18.91% 

4  Master’s/Doctorate  degree 

87 

6.07% 

5  Under  16  Inapplicable 

220 

15.35% 

Appendix  A  Univariate/bivariate  tables 


Table  19  Descriptive  statistics  of  insurance  characteristics 


Descriptives  (N=1433) 

Freq 

Percentage 

Insurance  Coverage  RD1-3 

1  Private 

976 

68.11 

2  Public 

364 

25.40 

3  Uninsured 

93 

6.49 

Insurance  Coverage  RD4-5 

1  Private 

893 

68.27 

2  Public 

314 

24.01 

3  Uninsured 

101 

7.72 

24  month  period  coverage 

1  Coverage  1 8-24  months 

1232 

85.97 

2  Coverage  12-17  months 

65 

4.54 

3  Coverage  6-11  months 

42 

2.93 

4  Coverage  0-6  months 

94 

6.56 
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Table  20  ICD9  Diagnostic  codes  of  Panel  1/Panel  3  respondents 


ICD-9  diagnosis 

Panel  1 

N=1067 

Panel  3 

N=557 

No  diagnosis 

111 

38 

Non-mental  health  diagnosis 

216 

99 

Mental  health  diagnosis 

740 

420 

295  Schizophrenic  disorder 

19 

9 

296  Affective  disorders 

31 

25 

297  Paranoid  states 

3 

298  Other  non-organic  disorders 

6 

2 

300  Neurotic  disorders 

173 

88 

301  Personality  disorders 

- 

3 

302  Sexual  disorders 

9 

4 

308  Acute  reactions  stress 

80 

59 

309  Adjustment  reactions 

13 

9 

311  Depressive  disorders 

351 

193 

V40  Mental/behavioral  problems 

22 

8 

V61  Other  family  circumstances 

20 

3 

V62  Other  psychosocial  circumstances 

13 

17 
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Table  21  Descriptive  statistics  of  impairment  characteristics 


Descriptive  (N=1433) 

Freq 

Percentage 

Perceived  health  status 

1  Excellent 

228 

15.97% 

2  Very  good 

354 

24.79% 

3  Good 

417 

29.20% 

4  Fair 

282 

19.75% 

5  Poor 

147 

10.29% 

Perceived  mental  health  status 

1  Excellent 

178 

12.46% 

2  Very  good 

306 

21.43% 

3  Good 

468 

32.77% 

4  Fair 

345 

24.16% 

5  Poor 

131 

9.17% 

Activity  limitations 

1  Yes 

332 

23.63% 

2  No 

1073 

76.37% 

Social  limitations 

1  Yes 

1226 

86.64% 

2  No 

189 

13.36% 
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Table  22  Bivariate  analysis  of  number  of  visits 

Average  number  of  visits  among  respondents  with  at  least  one  MH  visit  (N=1488) 


MEAN  visits  (SD) 

Gender 

1  Male 

13.06(20.79) 

2  Female 

13.01(21.58) 

Age 

2  5-17 

14.44  (20.27 

3  18-24 

9.07  (12.30) 

4  25-44 

14.41  (22.27) 

5  45-64 

13.80  (24.44) 

6  65-90 

6.23  (9.71) 

Race 

1  Hispanic 

12.31  (20.09) 

2  Black 

11.87  (20.37) 

3  Other 

13.30  (21.59) 

Education 

1  No  degree 

8.82  (12.54) 

2  HS/GED 

11.82  (18.43) 

3  Bachelor’s/Oth 

16.22  (27.29) 

4  Advanced  degree 

19.09  (33.40) 

5  Under  16  Inappl 

14.86  (33.40) 

Coverage  RD1-3 

1  Private 

12.05  (20.50) 

2  Public 

15.51  (23.50) 

3  Uninsured 

13.61  (19.34) 

Coverage  RD4-5 

1  Private 

12.62  (21.32) 

2  Public 

17.13  (24.46) 

3  Uninsured 

11.18(17.77) 

24  mo  coverage 

1  1 8-24  months 

13.22  (21.65) 

2  12-17  months 

12.38(21.51) 

3  6-11  months 

10.31  (12.56) 

4  0-6  months 

12.21  (19.16) 

General  diagnosis 

1  Mental  health 

14.55  (22.58) 

2  Medical  dx  only 

10.86  (19.32) 

3  No  diagnosis 

5.86  (9.80) 

Psychotropic  medication** 

No 

Yes 

Health  status  * 
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1  Excellent 

2  Very  good 

3  Good 

4  Fair 

5  Poor 

14.45  (26.95) 

10.80  (17.84) 

11.70  (16.75) 

16.24  (26.19) 

14.24  (19.24) 

Mental  health  ** 

1  Excellent 

7.52  (17.96) 

2  Very  good 

11.45  (21.92) 

3  Good 

13.43  (21.12) 

4  Fair 

15.04  (19.78) 

5  Poor 

17.92  (26.22) 

Activity  limitation  ** 

1  No 

11.52  (18.98) 

2  Yes 

17.80  (26.33) 

Social  limitations  * 

1  No 

12.01  (19.50) 

2  Yes 

19.43  (29.60) 

Provider  listens 

1  Yes 

13.07  (21.50) 

2  No 

12.40  (16.64) 

Provider  confidence 

1  Yes 

13.11  (21.60) 

2  No 

11.81  (16.70) 

Satisfaction  with  provider/ staff 

1  Very  satisfied 

13.00  (21.40) 

2  Somewhat 

12.70  (20.65) 

3  Not  too  satisfied 

15.72  (24.65) 

4  Not  at  all  satisfied 

12.78  (16.24) 

Satisfaction  with  quality  of  care 

1  Very  satisfied 

12.80  (21.76) 

2  Somewhat 

14.14  (20.10) 

3  Not  too  satisfied 

13.87  (19.70) 

4  Not  at  all  satisfied 

5.50  (7.08) 

**p<.01  *p<.05  F  statistic  used  to  evaluate  differences  in  means 
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Table  23  Mean  number  of  visits  for  depression  subgroup 


Average  number  of  visits  among  respondents  with  at  least  one  MH  visit  (N-1488) 


_ ^ - * - 

MEAN(SD) 

Gender 

1  Male 

16.82(24.16) 

2  Female 

16.00(23.70) 

Age 

2  5-17 

23.21(27.63) 

3  18-24 

13.25(16.83)) 

4  25-44 

18.65(22.95) 

5  45-64 

15.66(28.06) 

6  65-90 

8.08(11.39) 

Race 

1  Hispanic 

15.09(22.08) 

2  Black 

13.41(16.18) 

3  Other 

16.72(24.71) 

Education  * 

1  No  degree 

10.39(13.08) 

2  HS/GED 

15.29(20.88) 

3  Bachelor’s/Oth 

18.47(28.24) 

4  Advanced  degree 

24.03(36.19) 

5  Under  16  Inappl 

28.90(30.29) 

Coverage  RD1-3 

1  Private 

14.93(23.17) 

2  Public 

20.24(26.70) 

3  Uninsured 

14.03(16.31) 

Coverage  RD4-5 

1  Private 

14.93(23.35) 

2  Public 

21.82(27.38) 

3  Uninsured 

13.64(17.05) 

24  mo  coverage 

1  1 8-24  months 

16.78(24.75) 

2  12-17  months 

13.37(19.83) 

3  6-11  months 

13.09(17.44) 

4  0-6  months 

13.32(16.11) 

General  diagnosis 

1  Mental  health 

16.24(23.82) 

2  Medical  dx  only 

- 

3  No  diagnosis 

- 

Psychotropic  medication 

No 

15.80(31.33) 

Yes 

16.34(21.85) 

Health  status  * 

1  Excellent 

21.64(37.71) 
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2  Very  good 

11.90(14.97) 

3  Good 

14.06(18.03) 

4  Fair 

20.83(29.33) 

5  Poor 

15.03(15.84) 

Mental  health  * 

1  Excellent 

5.34(5.85) 

2  Very  good 

17.25(32.51) 

3  Good 

15.57(20.37) 

4  Fair 

17.16(21.47) 

5  Poor 

19.66(27.03) 

Activity  limitation 

1  No 

14.53(21.41) 

2  Yes 

lllMIHIh—i ■ — — 

Social  limitations  * 

1  No 

14.66(21.89) 

2  Yes 

23.60(29.90) 

Provider  listens 

1  Yes 

16.15(23.90) 

2  No 

18.27(22.27) 

Provider  confidence 

1  Yes 

16.22(23.98) 

2  No 

16.67(20.78) 

Satisfaction  with  provider/staff 

1  Very  satisfied 

16.02(24.13) 

2  Somewhat 

15.02(19.25) 

3  Not  too  satisfied 

24.4(35.05) 

4  Not  at  all  satisfied 

21.8(27.54) 

Satisfaction  with  quality  of  care 

1  Very  satisfied 

16.04(24.42) 

2  Somewhat 

16.42(20.95) 

3  Not  too  satisfied 

21.68(28.03) 

4  Not  at  all  satisfied 

4(1.22) 

**p<.01  *p<.05  F  statistic  used  to  evaluate  dil 

'ferences  in  means 
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Table  24  Partial  correlation  coefficients  showing  the  unique  variance  contributed  to  each  model 


Model  1 

Model  2 

Model  3 

More  than  one  visit 

More  than  two  visits 

More  than  three  visits 

Sex 

.0684  ** 

.0640  * 

.0516  * 

Education 

.0146  * 

.0467  * 

.0453  * 

Health 

.0518* 

.04* 

.0621  * 

Mental 

.0460  * 

Activity  limitations 

.0814** 

.0762  ** 

.0446  * 

Psychotropic  drugs 

.05* 

.0544  ** 

.0436  * 

Confidence  in  provider 

.0386  * 

Satisfaction  with  quality 

-.0570  ** 

Visit  weights 

*p<05  **p<01 
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Table  25  Main  effects  model  with  analytic  weights  of  the  relationship  between  satisfaction  with 
care  and  the  probability  of  more  than  one  visit 


F  statistic  (36,220) 

OR  (SE) 

95%  Cl 

Sex 

Male 

Female 

1.58(.34) 

1. 0-2.4 

Race 

White 

Hispanic 

1.0(.28) 

.58-1.7 

Black 

.47(.16)* 

.24-.91 

Age 

25-44 

18-24 

.89(37) 

.4-2.0 

45-64 

1.26(.25) 

.84-1.9 

65-90 

1.11(31) 

.63-1.9 

Education 

No  degree 

High  school/ GED 

1.15(31) 

.68-1.96 

Bachelor’s/ other  degree 

1.82(.68) 

.87-3.8 

Master’s/Doctorate 

2.0(1.03)** 

.71-5.5 

Insurance 

Private 

Public 

.95(.21) 

.62-1.5 

Uninsured 

.23  (.17)* 

.05-1.0 

Health 

Excellent 

Very  good 

.83(.29) 

.42-1.6 

Good 

1.2(.40) 

.61-2.3 

Fair 

1.7(.60) 

!  .83-3.4 

Poor 

2.4(1 .0)* 

1.04-5.4 

Mental  health 

Excellent 

Very  good 

1.42(.42) 

.8-2.5 

Good 

2.16(.61)** 

1. 2-3.8 

Fair 

1.8(.64) 

.87-3.6 

Poor 

1.34(37) 

.59-3.1 

Activity  limits 

No 

Yes 

1.94(.47)** 

1. 2-3.1 

Social  limits 

No 

Yes 

1.28(39) 

.71-2.32 

Psychotropic 

No 

Yes 

1.37(.27) 

.93-2.0 

Appt  difficulty 

Very  difficult 

No  difficulty 

.83(.17) 

.56-1.2 

Phone  difficulty 

Very  difficult 

No  difficulty 

.88(.18) 

.6-1.3 

Provider  listens 

No 

Yes 

.59  (.26) 

.25-1.4 

Confident  in 

No 

179 


Provider 

Yes 

2.8(1. 3)* 

1. 1-7.2 

Satisfaction  with 
provider/  staff 

Not  satisfied 

Satisfied 

2.0(.92) 

.82-4.95 

Satisfaction  w 
quality  of  care 

Not  satisfied 

Satisfied 

.32(.21) 

.085-1.18 

Probability  of  obtaining  an  F  statistic  at  least  as  large  at  random:  *p<.05  **p<.01 
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Appendix  B  Multivariate  tables 


Table  26  Stratified  multivariate  models  of  satisfaction  with  quality  of  care  on  probability  of  a 
subsequent  visit 


Satisfaction  with  quality  of  care 


More  than  1  visit 

More  than  2  visits 

More  than  3  visits 

Ref=Not  satisfied(O) 

Ref=Not  satisfied(O) 

Satisfied(l) 

Satisfied(l) 

Satisfied(l) 

OR(SE) 

OR(SE) 

OR(SE) 

Main  effects  LR  models 

0.23(14)* 

.51(24) 

.82(36) 

Stratified  by  sex 

Male 

Female 

.21  (.21 ) 

.11  (.11)* 

.23(19) 

.86(58) 

.19(16)* 

2.6(17) 

Stratified  by  race 
Hispanic 

Black 

Other 

.09(.14) 

.29(1.09) 

.14(.1 1  )* 

2.43(3.41) 

.25(49) 

.29(18)* 

19.38(30.45) 

.07(14) 

.48(27) 

Stratified  by  psdrug 

Yes 

No 

.51  (.53) 

,09(.08)* 

.27(24) 

.69(44) 

.94(73) 

.85(53) 

Stratified  by  diagnosis 
Mental  health 

Medical  only/  none 

.27(.22) 

.  1 5(.  1 6) 

.61(38) 

.34(29) 

1.22(70) 

.40(33) 

Stratified  by  health 
Excellent/very  good 
Good/fair/poor 

.13(18) 

.26(.19) 

.35(34) 

.84(48) 

.60(56) 

1.06(58) 

Stratified  by  agecat 
0-17/18-24 

25-44 

.53(84) 

.47(39) 

.38(50) 

1.56(103) 

.68(78) 

1.75(1.14) 

45-64 

45-64/65-90 

.007(01)** 

.05(06)* 

.23(23) 

Stratified  by  education 
Less  than  high  school 
High  school/GED 

College 

.62(82) 

.08(11)* 

.29(40) 

1.09(1.18) 

.54(40) 

.52(62) 

3.13(3.2) 

.82(55) 

.31(36) 

Stratified  by  coverage 
Private 

Public/Uninsured 

.26(16)* 

5.17(4.13)* 

Stratified  by  mental 
Excellent/very  good 
Good/fair/poor 

.06(08)* 

.32(22) 

.60(53) 

.52(29) 

1.8(15) 

.60(32) 

**  pc.Ol  *  p<.05 
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Appendix  B  Appendix  B  Multivariate  tables 


Table  27  Stratified  multivariate  models  of  the  relationship  between  satisfaction  with 
provider/staff  and  the  probability  of  a  subsequent  visit 


Satisfaction  with  provider/staff 


More  than  1  visit 

More  than  2  visits 

More  than  3  visits 

Ref=Not 

satisfied(O) 

Ref=Not 

satisfied(O) 

Ref=Not 

satisfied(O) 

Satisfied(l) 

Satisfied(l) 

Satisfied(l) 

OR(SE) 

OR(SE) 

OR(SE) 

Main  effects  LR 
models 

2.09(79) 

1.57(.54) 

1.01(34) 

Stratified  by  sex 

Male 

Female 

2.75(1.9) 

2.28(1.3) 

2.2(14) 

1.45(74) 

1.65(1.04) 

1.03(.52) 

Stratified  by  race 
Hispanic 

Black 

Other 

3.14(.322) 

.17(53) 

2.26(.97) 

1.02(.92) 

,56(.93) 

1.83(73) 

,46(.45) 

.60(1.03) 

1.32(.51) 

Stratified  by  psdrug 

Yes 

No 

1.87(1.18) 

3.4(2.04)* 

2.48(1.37) 

,94(.52) 

1.33(72) 

,70(.37) 

Stratified  by  diagnosis 
Mental  health 

Medical  only/  none 

4.02(2.04)** 

,82(.58) 

2.9(13)* 

,45(.30) 

1.71(76) 

.26(.18)* 

Stratified  by  health 
Excellent/very  good 
Good/fair/poor 

2.2(14) 

1.7(.81) 

3.53(.23)* 

,91(.39) 

3.54(2.42) 

.54(.23) 

Stratified  by  agecat 
0-17/18-24 

25-44 

45-64/65-90 

.91  (.68) 
2.18(1.29) 

7.53(70.8)* 

1.56(1.08) 

,54(.31) 

6.35(5.1)* 

,65(.42) 

,46(.26) 

193(1.30) 

Stratified  by  education 
Less  than  high  school 
High  school/GED 
College 

,52(.52) 

2.09(1.21) 

2.88(.323) 

.71  (.65) 
1.88(.97) 
.98(1.04) 

.  1 9(.  1 7) 
1.55(78) 
1.38(1.35) 

Stratified  by  coverage 
Private 

Public/Uninsured 

1.8(.82) 

3.3(2.3) 

0.269 

1.75(1.04) 

1.16(.50) 

,86(.52) 

Stratified  by  mental 
Excellent/very  good 
Good/fair/poor 

.88(.66) 

2.83(1.23)* 

.78(.55) 

1.76(.68) 

.91  (.66) 
1.04(.40) 

**p<.01  *p<.05 
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Appendix  B  Multivariate  tables 


Table  28  Stratified  multivariate  models  of  the  relationship  between  provider  listens  and  the 
probability  of  a  subsequent  visit 


Provider  listens 


More  than  1  visit 

More  than  2  visits 

More  than  3  visits 

msmf’. ' '  IKiuniii 

iiSTrilSIMi 

Ref=does  not 

listen(0) 

Provider  listens(l) 
OR(SE) 

Provider  listens(l) 
OR(SE) 

Provider  listens(l) 
OR(SE) 

Main  effects  models 

,85(.40) 

1.0(.40) 

1.05(.40) 

Stratified  by  sex 

Male 

.72(.65) 

1.79(1.24) 

2.19(1.5) 

Female 

.80(.44) 

,58(.31) 

By  race 

Hispanic 

,57(.89) 

.26(.42) 

,08(.13) 

Black 

1.3(2.09) 

,14(.19) 

.69(1.0) 

Other 

1.41  (.83) 

1.69(.81) 

1.4(.63) 

By  psychotropic 

Yes 

,79(.60) 

1.37(.82) 

.98(.56) 

No 

,78(.52) 

1.07(.66) 

1.13(.69) 

By  diagnosis 

Mental  health 

.50(.34) 

1.01  (.55) 

1.05(.49) 

Medical  only/  none 

1.91(1.6) 

1.35(1.01) 

2.03(1.6) 

By  health 

Excellent/very  good 

,64(.46) 

Good/fair/poor 

HHH 

1.23(.57) 

By  age  category 
0-17/18-24 

,64(.76) 

,72(.77) 

25-44 

1.23(.83) 

.95(.57) 

.82(.49) 

45-64/65-90 

,78(.79) 

1.42(1.12) 

2.2(1.54) 

By  education 

Less  than  HS 

1.32(  154) 

,82(.84) 

1.54(1.41) 

High  school/GED 

,56(.42) 

,92(.51) 

1.7(.60) 

College 

,42(.58) 

1.1(1.24) 

,57(,63) 

By  coverage 

Private 

,50(.31) 

1.14(.58) 

1.64(.79) 

Public/Uninsured 

3.06(2.5) 

1.33(.92) 

,93(.65) 

Stratified  by  mental 
Excellent/very  good 

,54(.45) 

,87(.60) 

1.08(.73) 

Good/fair/poor 

1.18(.73) 

,94(.47) 

1.02(.47) 

**p<.01  *p<.05 
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Appendix  B  Multivariate  tables 


Table  29  Stratified  multivariate  analysis  of  the  relationship  between  confidence  in  provider  and 
the  probability  of  a  subsequent  visit 


Confidence  in  provider 


More  than  1  visit  More  than  2  visits  More  than  3  visits 

Ref=Not  confident(O) 

Ref=Not  confident(O) 

Ref=Not  confident(O) 

■WHIM 

Confident  (1) 

Confident  (1) 

OR(SE) 

OR(SE) 

Main  effects  LR  models 

2.16(.89) 

1.35(.49) 

1.16(.41) 

Stratified  by  sex 

Male 

Female 

-  BESS 

1.05(65) 

1.53(.82) 

1.04(.62) 
1 .34(70) 

Stratified  by  race 
Hispanic 

Black 

Other 

4.89(5.72) 

.84(2.2) 

2.14(1.06) 

,65(.71) 
8.94(14.9) 
1.51  (.63) 

.96(1.03) 

1.25(2.23) 

1.40(.57) 

Stratified  by  psdrug 

Yes 

No 

1.93(1.43) 

2.28(1.33) 

1.5(.93) 

1.47(.80) 

1 .24(72) 
1.17(.63) 

Stratified  by  diagnosis 
Mental  health 

Medical  only/  none 

2.93(1.65) 
1.11  (.79) 

1.32(.64) 
1.21  (.78) 

1.23(.57) 

1.03(.68) 

Stratified  by  health 
Excellent/very  good 
Good/fair/poor 

1.94(1.21) 

2.81(1.58) 

2.02(1.78) 

.95(.46) 

Stratified  by  agecat 
0-17/18-24 

25-44 

45-64/65-90 

2.93(2.9) 

1.39(.71) 

1.78(1.56) 

1.78(1.68) 

1.48(73) 

1.0(.81) 

Stratified  by  education 
Less  than  high  school 
High  school/GED 

College 

10.6(12.4)* 
1.41  (.97) 
25.4(27.6)** 

5.34(5.25) 

,34(.21) 

7.9(6.2)** 

2.83(2.2) 

,412(.23) 

7.8(5.5)** 

Stratified  by  coverage 
Private 

Public/Uninsured 

2.27(1.03) 

,35(.26) 

1.98(.86) 

,30(.22) 

Stratified  by  mental 
Excellent/very  good 
Good/fair/poor 

1.25(.80) 

1.48(.68) 

,83(.52) 

1.43(.64) 

**p<.01  *p<.05 
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Appendix  B  Multivariate  tables 


Table  30  Stratified  multivariate  analysis  of  the  relationship  between  phone  difficulty  and  the 
probability  of  a  subsequent  visit 


Difficulty  contacting  provider  by  phone 


More  than  1  visit  More  than  2  visits  More  than  3  visits 

Ref=Difficult  (0) 

-  Vi-  -.-*  meSH 

■ 

WtBSSSmm 

Not  difficult  (1) 
OR(SE) 

Main  effects  LR  models 

,89(.06) 

.96(.16) 

1.122(.18) 

Stratified  by  sex 

Male 

Female 

.60(.22) 

1.05(.25) 

.74(.24) 

1.12(.24) 

.92(.29) 

1.15(.24) 

Stratified  by  race 

Hispanic 

Black 

Other 

.97(1.7) 

2.0(1.72) 

■77(.17) 

3.38(1.7)* 

.61  (.45) 
.86(.16) 

2.43(1.14) 

,70(.53) 

1 .51  (.21 ) 

Stratified  by  psdrug 

Yes 

No 

,99(.29) 

,90(.26) 

,90(.22) 

1.22(.33) 

,99(.23) 

1.3(.34) 

Stratified  by  diagnosis 
Mental  health 

Medical  only/  none 

,80(.19) 

1.38(.50) 

1.0(.19) 

1.9(.66) 

Stratified  by  health 
Excellent/very  good 
Good/fair/poor 

ms 

1.24(.32) 

1.06(.22) 

Stratified  by  agecat 
0-17/18-24 

25-44 

45-64/65-90 

1.40(.58) 

1.2(.36) 

,56(.19) 

1 

1.99(.74) 

,86(.22) 

1.19(.32) 

Stratified  by  education 

Less  than  high  school 

High  school/GED 

College 

1.3(.68) 
,94(.26) 
.71  (.33) 

2.08(.90) 

1.23(.30) 

,82(.28) 

Stratified  by  coverage 
Private 

Public/Uninsured 

hh 

1.22(.23) 

,99(.30) 

Stratified  by  mental 
Excellent/very  good 
Good/fair/poor 

,93(.27) 

.89(.21) 

1.25(.35) 

,93(.19) 

1.53(.44) 
1.03(.20)  . 

**p<.01  *p<.05 
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Appendix  B  Multivariate  tables 


Table  31  Stratified  multivariate  analysis  of  the  relationship  between  appointment  difficulty  and 
the  probability  of  a  subsequent  visit 


Difficulty  getting  appointment  with  provider 


More  than  1  visit  More  than  2  visits  More  than  3  visits 

Ref=Difficult(0) 

Ref=Difficult(0) 

Ref=Difficult(0) 

Not  difficult  (1) 
OR(SE) 

Not  difficult  (1) 
OR(SE) 

Not  difficult  (1) 
OR(SE) 

Main  effects  LR  models 

,97(.18) 

.94(15) 

1.04(.17) 

Stratified  by  sex 

Male 

Female 

,92(.33) 

1.10(.27) 

,82(.25) 

1.04(.23) 

1.14.35) 
1.21  (.25) 

Stratified  by  race 

Hispanic 

Black 

Other 

.36(.21) 

.12(.13) 

1.06(.23) 

.34(.17)* 

.34(.29) 

,98(.19) 

.59(.27) 

.89(73) 

1.04(.19) 

Stratified  by  psdrug 

Yes 

No 

,94(.26) 

1.06(.30) 

■HMSieSI 

1.11  (.26) 
1.19(32) 

Stratified  by  diagnosis 
Mental  health 

Medical  only/  none 

,97(.23) 

1.15(.43) 

.86(.18) 

,93(,32) 

1.07(21) 

.94(.34) 

Stratified  by  health 
Excellent/very  good 
Good/fair/poor 

,83(.23) 

,96(.24) 

,82(.22) 

1 . 1 3(.23) 

Stratified  by  agecat 
0-17/18-24 

25-44 

45-64/65-90 

.44(.22) 

,73(.23) 

1.56(.49) 

MM 

,48(.20) 

1.38(.36) 

1 . 1 5(.30) 

Stratified  by  education 

Less  than  high  school 

High  school/GED 

College 

,53(.28) 

1.26(.35) 

.67(.30) 

,48(.21) 

1.5(.37) 

.40(.15)* 

.83(.36) 
1.41  (.35) 
,73(.24) 

Stratified  by  coverage 
Private 

Public/Uninsured 

,92(.20) 

,87(.31) 

■H 

,84(.16) 

1.7(.50) 

Stratified  by  mental 
Excellent/very  good 
Good/fair/poor 

1.09(.32) 

.78(.19) 

,85(.24) 

,87(.18) 

1.19(34) 

,90(.18) 

**p<.01  *  p<.05 
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Appendix  B  Multivariate  tables 


Table  32  Stratified  multivariate  analysis  of  the  relationship  between  satisfaction  with  care  and 
the  probability  of  depression  treatment  compliance 


Quality  of 
care 

Ref=Not 
satisfied  (0) 

Provide 

r/staff 

Ref=Not 

satisfied 

(0) 

Provider 

listens 

Ref=does  not 
listen  (0) 

Confid  in 
provider 

Ref=Not 
confident  (0) 

Phone 

difficulty 

Ref=Difficul 

t(0) 

Appt 

difficulty 

Ref=DifficuI 

t(0) _ 

Satisfied(l) 

OR 

Satisfied( 

i) 

OR 

Provider 
listens  (1) 

OR 

Confident  (1) 
OR 

Not 

difficult  (1) 
OR 

Not 

difficult  (1) 
OR 

Main  effects 

,97(.63) 

1.2(.51) 

,66(.32) 

,85(.42) 

,78(.15) 

1.37(.47) 

By  sex 

Male 

Female 

BB 

,33(.41) 

1.2(.60) 

2.4(2.5) 

.73(.41) 

.02(.03)** 

.38(.24) 

.61  (.26) 
,75(.16) 

I 

wfmm 

By  race 
Hispanic 

Black 

Other 

m 

1.83(.98) 

.24(.25) 

.27(.18)* 

.68(.33) 

,72(.16) 

,90(.44) 

1.03(.23) 

By  psdrug 

Yes 

No 

,87(.47) 

27(35)* 

1.21  (.69) 
9.2(13.8) 

.08(.07)** 

,04(.08) 

m 

UM>I 

By  health 
Exc/very  good 
Good/fair/poor 

,33(.46) 

1.45(1.31) 

3.2(3. 0) 
1.1(6) 

1.7(16) 

,58(.39) 

.99(.92) 

,26(.19) 

.87(.28) 

.54(.14)* 

,68(.24) 

1.9(.5)* 

By  agecat 

0-17/18-24 

25-44 

45-64/65-90 

1.55(1.64) 

10.4(13.3) 

1.2(.80) 

1.4(13) 

1.8(11) 

,53(.55) 

.11  (.11)* 
.09(.10) 

,79(.24) 

,78(.23) 

,77(.24) 

1.64(.49) 

By  education 
Less  than  HS 
HS/GED 
College 

5.94(5.32)* 

.44(.56) 

1.22(76) 

5.4(7.4) 

.07(.06)** 

,24(.33) 

.73(.21) 

.56(.21) 

1.28(.38) 
.91  (.35) 

By  coverage 

Private 

Public/Uninsr 

2.2(1.55) 

1.91(3.6) 

1.9(10) 

.06(.08)* 

1.02(.62) 

.75(.65) 

.18(.11)** 

15(2.7) 

,98(.22) 

.38(.15)* 

,76(.18) 

8.3(3.7)** 

By  mental 
Exc/very  good 
Good/fair/poor 

2.73(4.39) 

1.94(1.34) 

2.9(3.1) 

.89(.48) 

106(1.12) 

,66(.38) 

.48(.43) 

.25(.17)* 

1.42(.50) 

.57(.13)* 

1.04(.37) 

1.45(.34) 

**p<.01  *p<.05 
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Table  33  Main  effects  model  using  analytic  weights  -  Relationship  between  satisfaction  with  care 
and  the  probability  of  depression  treatment  compliance 


Survey  logit  F  statistic  (df— 18  ) 

OR  (SE) 

Coefficient 

95%  Cl 

Race 

White 

mm  1 

Black 

-.87 

.14-1.28 

Age 

25-44 

0-17 

-.70 

.15-1.71 

18-24 

.01 

.37-2.81 

45-64 

mmEeM  ■ 

.02 

.51-2.03 

65-90 

.56  (.25) 

-.58 

.23-1.35 

Health 

Excellent 

Very  good 

1.65(.65) 

.50 

.75-3.6 

Good 

2.39(1.28) 

.87 

.83-6.88 

Fair 

3.12(1.71)* 

1.14 

1.05-9.22 

Poor 

ITTfTffflTUB 

1.57 

1.65-14.04 

Activity  limitations 

No 

mm 

Yes 

IllillyHI 

.51 

.82-3.4 

Appt  difficulty 

Very  difficult 

■mn 

No  difficulty 

.27 

.65-2.65 

Phone  difficulty 

mm 

1 

-.16 

.48-1.51 

Provider  listens 

wmmm 

Yes 

.55 

.49-6.06 

Confident  in  provid 

No 

wmmm 

Yes 

EisES9Hi 

-2.53 

.007-.90 

Satisfaction  with 

■mi 

provider/  staff 

.42 

.55-4.20 

Satisfaction  w 

Not  satisfied 

quality  of  care 

Satisfied 

4.37(5.37) 

1.47 

.39-49.54 

F(18, 144)=5.19 _ 

Probability  of  obtaining  a  F  statistic  at  least  as  large  at  random:  *p<.05  **p<.01 
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Figure  2  Adjusted  survival  curves:  high/  low  satisfaction  with  quality  of  care 


Figure  3  Adjusted  survival  curves:  high/  low  satisfaction  with  provider/staff 


analysis  time 


Appendix  C  Survival  analysis  figures  and  tables 


Figure  4  Adjusted  survival  curves:  has  confidence  (high)  /  does  not  have  confidence  (low) 


Figure  5  Adjusted  survival  curves:  provider  listens  (high)  /  provider  does  not  listen  (low) 
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Appendix  C  Survival  analysis  figures  and  tables 


Table  34  Stratified  Weibull  hazard  models  comparing  satisfaction  and  time-to-failure  rates 


Quality  of 
care 

Ref=Not  satisfied 

Provider  and 
staff 

Ref=Not  satisfied 

Provider 

listens 

Ref=does  not  listen 

Confidence  in 
provider 

Ref=Not  confident 

Satisfied  (1) 

Satisfied(l) 

Provider 
listens  (1) 

Confident  in 
provider(1) 

HR(SE) 

HR(SE) 

HR(SE) 

HR(SE) 

Main  effects  mode! 

1.29(.26) 

n 

Stratified  by  sex 
Male 

1.38(.40) 

•i  * 

Female 

1.13(.36) 

m  *1  PH  HH 

Stratified  by  race 
Hispanic 

,64(.33) 

1.61  (.60) 

1.5(.90) 

,82(.40) 

Black 

1.03(.72) 

1.17(.63) 

.39(.18)* 

1.9(1.22) 

Other 

1.42(.35) 

1 .12(.  18) 

■■HI 

.66(.12)* 

By  psychotropic 

Yes 

1.2(41) 

.83(.19) 

.87(22) 

No 

1.4(.37) 

1.43(.27) 

■K9@l 

-75(.  1 7) 

By  coverage 

Private 

1.42(.37) 

1.2(.20) 

1.7(.23) 

.70(.  14) 

Public 

,99(.49) 

.70(.24) 

,90(.31) 

1.4(61) 

Uninsured 

3.6(3.11) 

,43(.28) 

■ 

1.69(1.3) 

By  diagnosis 

Mental  health 

1.09(.29) 

.85(.15) 

,84(.18) 

Medical  only 

2.5(12) 

1.6(.50) 

,98(.38) 

No  diagnosis 

2.36(1.6) 

2.57(1.5) 

,34(.20) 

By  health 
(1-2) 

1.09(.40) 

.90(.22) 

■ 

.60(.15)* 

(3-5) 

1.58(.41) 

1.21  (.21) 

,78(.17) 

By  mental  health 
(1-2) 

1.19(.46) 

1.81  (.55)* 

1.54(.43) 

.49(.14)* 

(3-5) 

1.45(.35) 

,98(.16) 

.98(.20) 

,83(.17) 

By  age  category 
18-24 

2.9(2.6) 

,57(.30) 

.43(.51) 

1.59(1.5) 

25-44 

.81  (.24) 

1.18(.29) 

1.08(.27) 

,73(.16) 

45-64 

8.99(4.5)** 

.41  (.18)* 

,69(.25) 

.41  (.15)* 

65-90 

1.35(1.29) 

2.5(1.43) 

1.35(.98) 

,63(.64) 

By  education 

Less  than  HS 

1.28(.48) 

,98(.32) 

.83(.27) 

.76(.26) 

High  School/GED 

1.5(.51) 

■ 

1.14(.27) 

.71  (.1 9) 

Bachelors 

2.35(2.04) 

,52(.39) 

,68(.28) 

Advanced  degree 

63.7(79.6)** 

IHHi 

**p<.01  *p<.05 


Appendix  C  Survival  analysis  figures  and  tables 

Figure  6  Satisfaction  with  quality  of  care  (Age=  45-64)  (High=l  Low=0)  p<«01 


o  JquO=1  a  _lquO=0 
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Figure  7  Satisfaction  with  quality  of  care  (Diagnosis=medical  only)  (High=l  Low=0)  p<.05 


JquO=1  a  JquO=0 
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Figure  8  Satisfaction  with  quality  of  care  (Education=advanced  degree)  (High=l  Low=0)  p<.01 
O  Jqu0=1  A  JquO=0 
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Figure  9  Satisfaction  with  provider/staff  (Age=45-64)  (High=l  Low=0)  p<.05 
o  JpsO=1  a  JpsO=0 
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Figure  10  Satisfaction  with  provider/staff  (Education=advanced  degree)  (High-1  Low-0)  p<.05 
o  JpsO=1  A  JpsO=0 
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Figure  11  Satisfaction  with  provider/staff  (Mental  health-excellent/very  good)  p<.05 
o  JpsO=1  A  JpsO=0 
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Figure  12  Confidence  in  provider  (Health=excellent/very  good)  (Yes=l  No=0)  p<.05 
O  JcoO=1  a  JcoO=0 
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;ure  13  Confidence  in  provider  (Mental  health=excellent/very  good)  (Yes=l  No=0)  p<.  05 
o  JcoO=1  A  JcoO=0 
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Figure  14  Confidence  in  provider  (Age^  45-64)  (Yes=l  No=0)  p<.05 
O  lcoO=1  a  JcoO=0 


;ure  15  Confidence  in  provider  (Education=advanced  degree)  (Yes-1  No-0)  p<.01 
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Figure  16  Confidence  in  provider  (Race=  White)  (Yes=l  No=0)  p<.05 

o  lcoO=1  A  JcoO=0 


Figure  17  Confidence  in  provider  (Diagnosis=none)  (Yes=l  No=0)  p=.068 
o  lcoO=1  a  _lcoO=0 
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Figure  18  Provider  listens  (Education  =  advanced  degree)  (Yes=l  No=0)  p<.01 
o  lliOl  A  JliO=0 


*ure  19  Confidence  in  provider  Insurance=private  (Yes=l  No=0)  p=.07 
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Figure  20  Provider  listens  (Race=Black)  (Yes=l  No=0)  p<.05 
O  JliO=1  A  jno=o 
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Figure  21  Provider  listens  (Health=excellent/very  good)  (Yes=l  No=0)  p<.05 
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Figure  22  Appointment  difficulty  (Insurance=public)  (High=l  Low=0)  p<.  013 
o  JapO=1  a  _lapO=0 
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